Search NASA⌕ Search

SEARCH · Search NASA

Results for “false alarm rate”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

A Global Climatology of Extratropical Transition Part II: Statistical Performance of the Cyclone Phase Space

This study analyzes the differences between an objective, automated identification of tropical cyclones (TCs) that undergo extratropical transition (ET), and the designation of ET determined subjectively by human forecasters in best-track data in all basins globally. The objective identification of ET is based on the cyclone phase space (CPS), calculated from the Japanese 55-year Reanalysis (JRA-55) or the ECMWF Interim Reanalysis (ERA-Interim). The resulting classification into “ET storms” and “non-ET storms” underlies the global climatology of ET presented in Part I of this study. Here, the authors investigate how well the CPS classifications agree with those in the best-track records calculated from JRA-55 or from ERA-Interim data. According to F1 scores and Matthews correlation coefficients (MCCs), the classification of ET storms in the CPS agrees best with the best-track classification in the Western North Pacific (MCC > 0.7) and the North Atlantic (MCC > 0.5). In other basins, the correlation between the CPS classification and the best-track classification is only slightly higher than that of a random classification. The JRA-55 classification achieves higher performance scores than does the ERA-Interim classification, and the differences are statistically significant in all basins. The lower performance of ERA-Interim is mainly due to a higher false alarm rate, particularly in the Eastern North Pacific. Overall, the results show that while the CPS-based classifications are good enough to be useful for many purposes, there is almost certainly room for improvement – in the representation of the storms in reanalyses, in our objective metrics of ET, and in our scientific understanding of the ET process.

Melanie Bieli↗

Implementation Recommendations and Usage Boundaries for the Two-Dimensional Probability of Collision Calculation

The two-dimensional (2D) probability of collision (𝑃𝑐) estimation method relies on several assumptions that must be satisfied for accurate results. Monte Carlo analysis of ~44,000 conjunctions indicates that 2D-𝑃(sub 𝑐) pro-vides accurate estimates for most typical conjunctions, but occasionally underestimates 𝑃(sub 𝑐) significantly, indicating an assumption violation. A test to detect large-amplitude underestimation inaccuracies can be based on how much “offset-from-TCA” 2D-𝑃(sub 𝑐) values vary during a well-defined time interval bracketing closest approach. The test successfully detects all large-amplitude 2D-𝑃(sub 𝑐) underestimations found to date, but with a high false-alarm rate. The analysis also provides implementation recommendations and usage boundaries for the 2D-𝑃(sub 𝑐) method.

Hall, Doyle T.↗

Implementation Recommendations and Usage Boundaries for the Two-Dimensional Probability of Collision Calculation

The two-dimensional (2D) probability of collision (𝑃𝑐) estimation method relies on several assumptions that must be satisfied for accurate results. Monte Carlo analysis of ~44,000 conjunctions indicates that 2D-𝑃(sub 𝑐) pro-vides accurate estimates for most typical conjunctions, but occasionally underestimates 𝑃(sub 𝑐) significantly, indicating an assumption violation. A test to detect large-amplitude underestimation inaccuracies can be based on how much “offset-from-TCA” 2D-𝑃(sub 𝑐) values vary during a well-defined time interval bracketing closest approach. The test successfully detects all large-amplitude 2D-𝑃(sub 𝑐) underestimations found to date, but with a high false-alarm rate. The analysis also provides implementation recommendations and usage boundaries for the 2D-𝑃(sub 𝑐) method.

Hall, Doyle T.↗

A Touchpad-Based Method for Inducing Attentional Tunneling

Attentional tunneling is a recognized problem for aviation safety in the flight deck. A prototype system (touchpad and associated application and experimental software) was developed and evaluated for its success in inducing attentional tunneling in a reliable and predictable manner in training and experimental contexts. Two experiments involving a total of sixteen participants examined baseline performance for visual memory of a color or number sequence, simultaneous with performing a competing auditory detection task. Spatial auditory separation of the auditory stimuli was also evaluated. Data are provided for various aspects of touchpad entry (accuracy, speed) as well as hit and false alarm rates for the auditory task. The results will help determine means of inducing attentional tunneling in more complex flight simulator experiments, and for developing an inexpensive prototype for pilots to measure cognitive fixation and develop mitigation strategies.

aviation safety↗

The International Space Station Lightning Imaging Sensor: Performance, Applications, and Cross-Platform Science

The Lightning Imaging Sensor (LIS), an optically based lightning detector developed at NASA Marshall Space Flight Center, was launched to the International Space Station (ISS) in February 2017, detecting optical signatures of lightning with storm-scale horizontal resolution (4-km) during both day and night. ISS LIS data are available 1 March 2017 to present. Millisecond timing allows detailed intercalibration and validation with other spaceborne and ground-based lightning sensors. Initial comparisons with those other sensors suggest flash detection efficiency around 60% (diurnal variability of 51-75%), false alarm rate under 5%, timing accuracy better than 2 ms, and horizontal location accuracy around 3 km. The spatially uniform flash detection capability of ISS LIS from low-Earth orbit allows assessment of spatially varying flash detection efficiency for other sensors and networks, particularly the Geostationary Lightning Mappers (GLMs). ISS LIS provides research data suitable for investigations of lightning physics, climatology, thunderstorm processes, and atmospheric composition, as well as realtime lightning data for operational forecasting and aviation weather interests. ISS LIS enables enrichment and extension of the long-term global climatology of lightning from space, and is the only recent platform that extends the global record to higher latitudes (± 55°). The global spatial distribution of lightning from ISS LIS is broadly similar to previous datasets, with globally averaged seasonal/annual flash rates (3-year average: ~44 s) about 5-10% lower. This difference is likely due to reduced flash detection efficiency that will be mitigated in future ISS LIS data processing, as well as the shorter ISS LIS period of record. The expected land/ocean contrast in the diurnal variability of global lightning is also observed. The near-realtime ISS LIS data make the instrument important to operational weather forecasting and related applications, including public safety. Finally, ISS LIS is demonstrating utility as part (or potential part) of cross-platform studies examining a diverse array of topics, including lightning physics, thunderstorm processes, convective precipitation, and atmospheric composition.

Timothy Lang↗

GROWTH on S190814bv: Deep Synoptic Limits on the Optical/Near-infrared Counterpart to a Neutron Star–Black Hole Merger

On 2019 August 14, the Advanced LIGO and Virgo interferometers detected the high-significance gravitational wave (GW) signal S190814bv. The GW data indicated that the event resulted from a neutron star–black hole (NSBH) merger, or potentially a low-mass binary BH merger. Due to the low false-alarm rate and the precise localization (23 deg^(2) at 90%), S190814bv presented the community with the best opportunity yet to directly observe an optical/near-infrared counterpart to an NSBH merger. To search for potential counterparts, the GROWTH Collaboration performed real-time image subtraction on six nights of public Dark Energy Camera images acquired in the 3 weeks following the merger, covering >98% of the localization probability. Using a worldwide network of follow-up facilities, we systematically undertook spectroscopy and imaging of optical counterpart candidates. Combining these data with a photometric redshift catalog, we ruled out each candidate as the counterpart to S190814bv and placed deep, uniform limits on the optical emission associated with S190814bv. For the nearest consistent GW distance, radiative transfer simulations of NSBH mergers constrain the ejecta mass of S190814bv to be M(ej) < 0.04 Mꙩ at polar viewing angles, or M(ej) < 0.03 Mꙩ if the opacity is κ < 2 sq.cm/g. Assuming a tidal deformability for the NS at the high end of the range compatible with GW170817 results, our limits would constrain the BH spin component aligned with the orbital momentum to be χ < 0.7 for mass ratios Q < 6, with weaker constraints for more compact NSs.

Igor Andreoni↗

GWTC-1: A Gravitational-Wave Transient Catalog of Compact Binary Mergers Observedby LIGO and Virgo During the First and Second Observing Runs

We present the results from three gravitational-wave searches for coalescing compact binaries with component masses above1M during the first and second observing runs of the advanced gravitational-wave detector network. During the first observing run (O1), from September 12, 2015 to January 19, 2016, gravitational waves from three binary black hole mergers were detected. The second observing run (O2), which ran from November 30, 2016 to August 25, 2017, saw the first detection of gravitational waves from a binary neutron star in spiral, in addition to the observation of gravitational waves from a total of seven binary black hole mergers, four of which we report here for the first time: GW170729, GW170809, GW170818, and GW170823. For all significant gravitational-wave events, we provide estimates of the source properties. The detected binary black holes have total masses between18.6þ3.2−0.7M and 84.4þ15.8−11.1M and range in distance between 320þ120−110 and 2840þ1400−1360Mpc. No neutron star–black hole mergers were detected. In addition to highly significant gravitational-wave events, we also provide a list of marginal event candidates with an estimated false-alarm rate less than 1 per 30 days. From these results over the first two observing runs, which include approximately one gravitational-wave detection per 15 days of data searched, we infer merger rates at the 90% confidence intervals of 110−3840Gpc−3y−1 for binary neutron stars and 9.7−101Gpc−3y−1 for binary black holes assuming fixed population distributions and determine a neutron star–black hole merger rate 90% upper limit of 610Gpc−3y−1.

Astrophysics↗

GW190521: A Binary Black Hole Merger with a Total Mass of 150 Mꙩ

On May 21, 2019 at 03:02:29 UTC Advanced LIGO and Advanced Virgo observed a short duration gravitational-wave signal, GW190521, with a three-detector network signal-to-noise ratio of 14.7, and an estimated false-alarm rate of 1 in 4900 yr using a search sensitive to generic transients. If GW190521 is from a quasicircular binary inspiral, then the detected signal is consistent with the merger of two black holes with masses of 85 (+21,−14) Mꙩ and 66 (+17,−18) Mꙩ (90% credible intervals). We infer that the primary black hole mass lies within the gap produced by (pulsational) pair-instability supernova processes, with only a 0.32% probability of being below 65 M⊙. We calculate the mass of the remnant to be 142 (+28,−16) Mꙩ, which can be considered an intermediate mass black hole (IMBH). The luminosity distance of the source is 5.3 (+2.4,−2.6) Gpc, corresponding to a redshift of 0.82 (+0.28,−0.34) . The inferred rate of mergers similar to GW190521 is 0.13 (+0.30,−0.11) Gpc^(−3) per yr.

R. Abbott↗

Using Satellite Soil Moisture and Rainfall in the Landslide Hazard Assessment for Situational Awareness System

The Landslide Hazard Assessment for Situational Awareness system(LHASA)gives a global view of landslide hazard in nearly real time. Currently, it is being upgraded from version 1 to version 2, which entails improvements along several dimensions. These include the incorporation of new predictors, machine learning, and new event-based landslide inventories. As a result, LHASA version 2 substantially improves on the prior performanceand introduces a probabilistic element to the global landslide nowcast. Data from the soil moisture active-passive (SMAP) satellite has been assimilated into a globally consistent data product with a latency less than 3 days, known as SMAP Level 4. In LHASA, thesedata representthe antecedent conditions prior to landslide-triggering rainfall. In some cases, soil moisture may have accumulated over aperiod of many months. The model behind SMAP Level 4 also estimates the amount of snow on the ground, which is an important factor in some landslide events. LHASA also incorporates this information as an antecedent condition that modulates the response torainfall. Slope, lithology, and active faults were also used as predictor variables. These factors can have a strong influence on where landslides initiate.LHASA relies on precipitation estimates from the Global Precipitation Measurement mission to identify the locations where landslides are most probable. The low latency and consistent global coverage of these data make them ideal for real-time applications at continental to global scales. LHASA relies primarily on rainfall from the last 24 hours to spothazardous sites, which is rescaled by the local 99thpercentile rainfall.However, the multi-day latency of SMAP requires the use of a 2-day antecedent rainfall variable to represent the accumulation of rain between the antecedent soil moisture and current rainfall. LHASA merges these predictors with XGBoost, a commonly used machine-learning tool, relying on historical landslide inventories to develop the relationship between landslide occurrence and various risk factors. The resulting model relies heavily on current daily rainfall, but other factors also play an important role. LHASA outputsthe probability oflandslide occurrence ona grid of roughly one kilometer over all continents from 60 North to 60 South latitude. Evaluation over the period 2019-2020 showsthat LHASA version 2 doubles the accuracy of the global landslide nowcast without increasing the global false alarm rate. LHASA also identifies the areas where the human exposure to landslide hazard is most intense. Landslide hazard is divided into 4 levels: minimal, low, moderate, and high. Next, the number of persons and the length of major roads (primary and secondary roads)within each of these areas is calculated for every second-level administrative district (county). These results can be viewedthrough a web portal hosted at the Goddard Space Flight Center. In addition, users can download daily hazard and exposure data.LHASAversion 2uses machine learning and satellite data to identify areas of probable landslide hazard within hours of heavy rainfall. Itsglobal maps are significantly more accurate, and it now includes rapid estimates of exposed populations and infrastructure. In addition, a forecast mode will be implemented soon.

Thomas Stanley↗

Swift-XRT Follow-up of Gravitational-wave Triggers in the Second Advanced LIGO/Virgo Observing Run

The Neil Gehrels Swift Observatory carried out prompt searches for gravitational-wave (GW) events detected by the LIGO/Virgo Collaboration (LVC) during the second observing run ("O2"). Swift performed extensive tiling of eight LVC triggers, two of which had very low false-alarm rates (GW170814 and the epochal GW170817), indicating a high confidence of being astrophysical in origin; the latter was the first GW event to have an electromagnetic counterpart detected. In this paper we describe the follow-up performed during O2 and the results of our searches. No GW electromagnetic counterparts were detected; this result is expected, as GW170817 remained the only astrophysical event containing at least one neutron star after LVC's later retraction of some events. A number of X-ray sources were detected, with the majority of identified sources being active galactic nuclei. We discuss the detection rate of transient X-ray sources and their implications in the O2 tiling searches. Finally, we describe the lessons learned during O2 and how these are being used to improve the Swift follow-up of GW events. In particular, we simulate a population of gamma-ray burst afterglows to evaluate our source ranking system's ability to differentiate them from unrelated and uncataloged X-ray sources. We find that ≈60%–70% of afterglows whose jets are oriented toward Earth will be given high rank (i.e., "interesting" designation) by the completion of our second follow-up phase (assuming that their location in the sky was observed), but that this fraction can be increased to nearly 100% by performing a third follow-up observation of sources exhibiting fading behavior.

N. J. Klingler↗

Population Properties of Compact Objects from the Second LIGO-Virgo Gravitational-Wave Transient Catalog

We report on the population of 47 compact binary mergers detected with a false-alarm rate of <1yr^(−1) in the second LIGO–Virgo Gravitational-Wave Transient Catalog. We observe several characteristics of the merging binary black hole (BBH) population not discernible until now. First, the primary mass spectrum contains structure beyond a power law with a sharp high-mass cutoff; it is more consistent with a broken power law with a break at 39.7^(+20.3)(-9.1)M or a power law with a Gaussian feature peaking at 33.1^(+4.0)(-5.6)M (90% credible interval). While the primary mass distribution must extend to ~65M or beyond, only 2.9^(+3.5)(-1.7)M% of systems have primary masses greater than 45M. Second, we find that a fraction of BBH systems have component spins misaligned with the orbital angular momentum, giving rise to precession of the orbital plane. Moreover, 12% to 44% of BBH systems have spins tilted by more than 90°, giving rise to a negative effective inspiral spin parameter, χeff. Under the assumption that such systems can only be formed by dynamical interactions, we infer that between 25% and 93% of BBHs with nonvanishing |χeff| > 0.01 are dynamically assembled. Third, we estimate merger rates, finding RBBH = 23.9^(+14.3)(-8.6) Gpc^(-3) yr^(-1) for BBHs and RBNS = 320^(+490)(-240) Gpc^(-3) yr^(-1) for binary neutron stars. We find that the BBH rate likely increases with redshift (85% credibility) but not faster than the star formation rate (86% credibility). Additionally, we examine recent exceptional events in the context of our population models, finding that the asymmetric masses of GW190412 and the high component masses of GW190521 are consistent with our models, but the low secondary mass of GW190814 makes it an outlier.

R Abbott↗

GWTC-2: Compact Binary Coalescences Observed by LIGO and Virgo During the First Half of the Third Observing Run

We report on gravitational wave discoveries from compact binary coalescences detected by Advanced LIGO and Advanced Virgo in the first half of the third observing run (O3a) between 1 April 2019 15:00 UTC and 1 October 2019 15:00 UTC. By imposing a false-alarm-rate threshold of two per year in each of the four search pipelines that constitute our search, we present 39 candidate gravitational wave events. At this threshold, we expect a contamination fraction of less than 10%. Of these, 26 candidate events were reported previously in near real-time through GCN Notices and Circulars; 13 are reported here for the first time. The catalog contains events whose sources are black hole binary mergers up to a redshift of ~ 0.8, as well as events whose components could not be unambiguously identified as black holes or neutron stars. For the latter group, we are unable to determine the nature based on estimates of the component masses and spins from gravitational wave data alone. The range of candidate event masses which are unambiguously identified as binary black holes (both objects ≥ 3 M⨀) is increased compared to GWTC-1, with total masses from ∼ 14M⨀ for GW190924 021846 to ∼ 150M⨀ for GW190521. For the first time, this catalog includes binary systems with significantly asymmetric mass ratios, which had not been observed in data taken before April 2019. We also find that 11 of the 39 events detected since April 2019 have positive effective inspiral spins under our default prior (at 90% credibility), while none exhibit negative effective inspiral spin. Given the increased sensitivity of Advanced LIGO and Advanced Virgo, the detection of 39 candidate events in ∼26 weeks of data (∼1.5 per week) is consistent with GWTC-1.

Gravitational wave detection↗

A New ML-Based Adaptive Thinning Methodology to Improve the Impact of AIRS and CrIS Assimilation on Global Tropical Cyclone Forecasts

This work builds on previous research performed by this team to improve the forecast of Tropical Cyclones (TCs) by assimilating AIRS and CrIS radiances into the NASA Global Earth Observing System (GEOS). Past published work demonstrated that the assimilation of radiances with variable density was beneficial to TC forecasting in the GEOS. In the previous setup, a fixed-size moving square named 'TC domain' was activated by the so-called TC-vitals, an international real-time message accessible to all NWP forecasting centers, that documents the existence of a TC, its estimated position, and its size. The information from TC-vitals activated a switch in the GEOS, which allowed to reduce the distance used for thinning AIRS and CrIS data inside a 15 degrees by 15 degrees moving TC domain centered on the storm, so that more data were assimilated in the vicinity of the TC during its lifetime. The methodology produced improved TC analyses and led to better forecasts, particularly related to intensity, without damaging the global forecast skill. In the new version, the adaptive thinning methodology is based on a machine-learning technique. The technique searches for TCs and creates TC masks by using cloud-top temperatures from all geostationary satellites without the need for additional information. It is being trained against the International Best Track Archive for Climate Stewardship (IBTrACS) data base. Once a TC mask is created, a switch identical to the one used in the previous adaptive thinning method is activated, allowing the GEOS to ingest more data in the TC-shaped size-changing domain that follows the storm. As of today, the team has been able to successfully assimilate data inside the ML-detected TC domains. Future work includes an improved capability of reducing false alarm rates (i.e., cloud systems that are erroneously labeled as TCs).

Oreste Reale↗

Setting the Bar for the Replacement of the Probability of Collision Metric

To date, satellite conjunction assessment (CA) risk analysis has largely embraced the probability of collision (Pc) as the omnibus metric to evaluate collision likelihood, and its use in such assessments has mostly been straightforward: at the point at which a conjunction mitigation decision is required, the calculated Pc is compared to a threshold; and if the calculated Pc exceeds that threshold, then a mitigation action is warranted. With only minor variation, this approach is employed by major CA risk assessment centers (e.g., NASA, EUSST, CNES, JAXA) and is advanced as the preferred method in the published CA best practices handbooks. Despite this near unanimity of operational practice, there is a major strain of secondary literature critical of the Pc and willing to propose alternatives. Alfano (2005) pointed out the ability of the Pc to underrepresent the risk in certain situations and counselled a maximum Pc construct. Carpenter (2017, 2019) reiterated this criticism and proposed using instead a confidence interval on the miss distance. Balch et al. (2019) identified what they argued was a defect in the entire Bayesian Pc construct and believed that the use of a more conservative methodology based on covariance ellipsoid overlap was necessary. Delande (2022) introduced the framework of collision “plausibility” to the risk assessment process and sketched out how this might be used operationally. Elkantassi (2022) published a full development of the miss distance confidence interval approach and applied it to several worked examples. While these different approaches to collision risk assessment do differ in their details, they all converge on two central points: first, the Pc’s failure to give an adequate expression of the risk in dilution region situations is a fatal flaw; and second, a conjunction should be presumed risky and in need of mitigation until the evidence of the situation can establish otherwise. These criticisms, if correct, would counsel a number of modifications to current CA operational practice; as such, they force a re-examination of fundamental aspects of the CA problem, including the following: 1. Is the CA risk assessment a probability problem, a statistics problem, or something else? 2. If it is a statistics problem, does it lend itself naturally to a hypothesis test construction? 3. If it can be construed as a hypothesis test, what form should the null hypothesis take, to wit: what constraints exist on the choice of the null hypothesis, what selections are in best alignment with all of the attendant parameters of the problem, and what is implied philosophically by different choices? 4. What are the implications of using the different proposed risk assessment parameters for CA? This question should be answered both in determining how frequently the dilution region situation cited by the critics of the Pc actually appears in an operationally significant manner and the missed detection and false alarm rates of all of the proposed risk assessment metrics, compared both to the Pc and to each other. This paper explores and offers preliminary answers to the above questions, presenting a researched treatment of the philosophical nature of the CA problem and the null hypothesis choice that achieves the greatest consistency with all of the different aspects of operational CA conduct. It then profiles all of the different proposed risk assessment metrics enumerated in the earlier paragraph against an extremely large database of conjunction events at both the 550km and 700km altitudes. The combination of the philosophical exploration of the CA problem and the results of the profiling activity articulates what a risk assessment metric will need to demonstrate, in terms of both innate construction and performance, in order to be a true competitor to the Pc.

conjunction assessment↗

Multi-Agency Ensemble Forecast of Wildfire Air Quality in the United States: Toward Community Consensus of Early Warning

Wildfires pose increasing risks to human health and properties in North America. Due to large uncertainties in fire emission, transport, and chemical transformation, it remains challenging to accurately predict air quality during wildfire events, hindering our collective capability to issue effective early warnings to protect public health and welfare. Here we present a new real-time Hazardous Air Quality Ensemble System (HAQES) by leveraging various wildfire smoke forecasts from three U.S. federal agencies (NOAA, NASA, and Navy). Compared to individual models, the HAQES ensemble forecast significantly enhances forecast accuracy. To further enhance forecasting performance, a weighted ensemble forecast approach was introduced and tested. Compared to the unweighted ensemble mean, the multilinear regression weighted ensemble reduced fractional bias by 34% in the major fire regions, false alarm rate by 72%, and increased hit rate by 17%. Finally, we improved the weighted ensemble using quantile regression and weighted regression methods to enhance the forecast of extreme air quality events. The advanced weighted ensemble increased the PM2.5 exceedance hit rate by 55% compared to the ensemble mean. Our findings provide insights into the development of advanced ensemble forecast methods for wildfire air quality, offering a practical way to enhance decision-making support to protect public health.

Yunyao Li↗

Early Fire Detection System for Partial and Low Gravity Environments

We present preliminary results on the development of a fire detection system for partial and low gravity environments. A significant challenge is the design of a system with a low false alarm rate. The detection system therefore has to discriminate between combustion particles, cabin, and lunar dust. Aerosol properties are measured and cross referenced to the target detectors at the Gases and Aerosols from Smoldering Polymers (GASP) Laboratory located at the NASA Glenn Research Center. A computational fluid dynamics plume model is also employed to determine the smoke transport properties of the targeted habitats.

Early fire detectioin↗

Study of Classifiers for U-235 Source Signatures Using Gamma Spectral Measurements

Signatures associated with low-level U-235 sources are studied from a classification analytics perspective, using NaI gamma-ray spectral measurements from detectors located at various distances from the source. Data sets collected at a shielded facility are utilized, wherein the source is introduced via a conduit into a formation of 21 NaI detectors deployed over 6 x 6 meters area in a formation of two concentric circles and one spiral. The activity levels in the spectral regions associated with potential U-235 signatures are estimated as counts at 1 second intervals, and are used as features to train classifiers for detecting the presence of the source. Eight different classifiers are trained and tested using the background and source measurements collected over multiple experimental runs. As expected, the classifier performance improved overall as measurements from the detectors closer to source are used, but also revealed unexpectedly low performance by two detectors that are identically produced and configured as others. Six of eight classifiers have an overall comparable performance, for example, three of them achieved zero training error and 99% detection at 4% false alarm rate for a detector located 1.3 meters away from the source. Also, larger training sets led to improved classification performance across all classifiers, and interestingly, the classifiers with the minimum training error did not necessarily achieve the highest classification performance on test data.

Rao, Nageswara↗

Using High-Resolution NSRDB Data to Evaluate Cloud Mask Forecast from WRF-Solar EPS

Validating spatiotemporal distributions of cloud forecasts using numerical weather prediction (NWP) models is difficult as this requires high-quality cloud-property at significantly high spatial and temporal resolution over extended periods of time. Observations of cloud properties, such as cloud mask, cloud optical thickness, and cloud type, are vital for assessing the capability of NWP models to forecast various types of clouds. Using the National Solar Radiation Database (NSRDB), this research evaluates ensemble cloud-mask predictions from the WRF-Solar ensemble prediction system (WRF-Solar EPS). From the WRF-Solar EPS, day-ahead solar forecasts for the contiguous United States (CONUS) for 2018 are simulated. Given the NSRDB data is accessible at a resolution of 2 km, we can calculate the cloud fraction across the 9-km grid of WRF-Solar EPS. This allows us to spatially assess the cloud-mask forecasts using two methods against the high-resolution NSRDB: (a) considering all 2-km NSRDB clouds in the forecast domain (EMAll), and (b) using a minimum cloud fraction threshold of 50% to designate a pixel as cloudy (EMP50). The low-resolution cloud masks from WRF-Solar EPS are evaluated directly against the cloud-resolving scale gridded observations from NSRDB using EMAll. With EMP50, we presume that scenes with less than 50% cloud cover from the 2-km NSRDB are clear. Thus, this assessment approach allows for a fair comparison with WRF-Solar EPS resolved to a 9-km grid. A method of point-by-point verification is used to evaluate dichotomous (yes/no) cloud mask predictions against the NSRDB. For each pixel of model extent, cloud frequency and traditional metrics (e.g., probability of detection, false alarm rate, and hit rate, etc.) are computed and compared with satellite-derived data sets. Mismatched cloud frequency (MCF) is computed to measure the present capability of WRF-Solar EPS in representing various types of clouds, which are categorized using three levels of cloud top height (CTH) and cloud optical depth (COD) across entire CONUS. Preliminary results show that the WRF-Solar EPS provides MCF values ranging from 9% to 46%, 16% to 33%, and 8% to 27% for low-level, middle-level, and high-level clouds, respectively, for three CTHs. The model produces MCFs ranging from 27% to 46%, 13% to 34%, and 8% to 19% for thin, medium-thickness, and thick clouds, respectively, for three CODs. The presentation will include a detailed description of the current outcomes as well as potential future extensions. The evaluation approach established in this study is readily extensible to the evaluation of cloud predictions from different ensemble NWP models. In addition, the findings of the suggested evaluation technique aid in identifying model weaknesses and will ultimately lead to advances in WRF-Solar EPS's skill in predicting clouds and solar irradiance.

cloud mask forecast↗