Search NASA⌕ Search

SEARCH · Search NASA

Results for “forecast evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

Ensemble Statistical Post-Processing of the National Air Quality Forecast Capability: Enhancing Ozone Forecasts in Baltimore, Maryland

An ensemble statistical post-processor (ESP) is developed for the National Air Quality Forecast Capability (NAQFC) to address the unique challenges of forecasting surface ozone in Baltimore, MD. Air quality and meteorological data were collected from the eight monitors that constitute the Baltimore forecast region. These data were used to build the ESP using a moving-block bootstrap, regression tree models, and extreme-value theory. The ESP was evaluated using a 10-fold cross-validation to avoid evaluation with the same data used in the development process. Results indicate that the ESP is conditionally biased, likely due to slight overfitting while training the regression tree models. When viewed from the perspective of a decision-maker, the ESP provides a wealth of additional information previously not available through the NAQFC alone. The user is provided the freedom to tailor the forecast to the decision at hand by using decision-specific probability thresholds that define a forecast for an ozone exceedance. Taking advantage of the ESP, the user not only receives an increase in value over the NAQFC, but also receives value for An ensemble statistical post-processor (ESP) is developed for the National Air Quality Forecast Capability (NAQFC) to address the unique challenges of forecasting surface ozone in Baltimore, MD. Air quality and meteorological data were collected from the eight monitors that constitute the Baltimore forecast region. These data were used to build the ESP using a moving-block bootstrap, regression tree models, and extreme-value theory. The ESP was evaluated using a 10-fold cross-validation to avoid evaluation with the same data used in the development process. Results indicate that the ESP is conditionally biased, likely due to slight overfitting while training the regression tree models. When viewed from the perspective of a decision-maker, the ESP provides a wealth of additional information previously not available through the NAQFC alone. The user is provided the freedom to tailor the forecast to the decision at hand by using decision-specific probability thresholds that define a forecast for an ozone exceedance. Taking advantage of the ESP, the user not only receives an increase in value over the NAQFC, but also receives value for

ozone↗

Objective Lightning Forecasting at Kennedy Space Center/Cape Canaveral Air Force Station using Cloud-to-Ground Lightning Surveillance System Data

The 45th Weather Squadron (45 WS) forecasters at Cape Canaveral Air Force Station (CCAFS) in Florida include a probability of thunderstorm occurrence in their daily morning briefings. This information is used by personnel involved in determining the possibility of violating Launch Commit Criteria, evaluating Flight Rules for the Space Shuttle, and daily planning for ground operation activities on Kennedy Space Center (KSC)/CCAFS. Much of the current lightning probability forecast is based on a subjective analysis of model and observational data. The forecasters requested that a lightning probability forecast tool based on statistical analysis of historical warm-season (May - September) data be developed in order to increase the objectivity of the daily thunderstorm probability forecast. The tool is a set of statistical lightning forecast equations that provide a lightning occurrence probability for the day by 1100 UTC (0700 EDT) during the warm season. This study used 15 years (1989-2003) of warm season data to develop the objective forecast equations. The local CCAFS 1000 UTC sounding was used to calculate stability parameters for equation predictors. The Cloud-to-Ground Lightning Surveillance System (CGLSS) data were used to determine lightning occurrence for each day. The CGLSS data have been found to be more reliable indicators of lightning in the area than surface observations through local informal analyses. This work was based on the results from two earlier research projects. Everitt (1999) used surface observations and rawinsonde data to develop logistic regression equations that forecast the daily thunderstorm probability at CCAFS. The Everitt (1999) equations showed an improvement in skill over the Neumann-Pfeffer thunderstorm index (Neumann 1971), which uses multiple linear regression, and also persistence and climatology forecasts. Lericos et al. (2002) developed lightning distributions over the Florida peninsula based on specific flow regimes. The flow regimes were inferred from the average wind direction in the 1000-700 mb layer at Miami (MIA), Tampa (TBW), and Jacksonville (JAX), Florida, and the lightning data were from the National Lightning Detection Network. The results suggested that the daily flow regime may be an important predictor of lightning occurrence on KSC/CCAFS.

Lambert, Winifred↗

An Integrated 0-1 Hour First-Flash Lightning Nowcasting, Lightning Amount and Lightning Jump Warning Capability

Using satellite-based methods that provide accurate 0-1 hour convective initiation (CI) nowcasts, and rely on proven success coupling satellite and radar fields in the Corridor Integrated Weather System (CIWS; operated and developed at MIT-Lincoln Laboratory), to subsequently monitor for first-flash lightning initiation (LI) and later period lightning trends as storms evolve. Enhance IR-based methods within the GOES-R CI Algorithm (that must meet specific thresholds for a given cumulus cloud before the cloud is considered to have an increased likelihood of producing lightning next 90 min) that forecast LI. Integrate GOES-R CI and LI fields with radar thresholds (e.g., first greater than or equal to 40 dBZ echo at the -10 C altitude) and NWP model data within the WDSS-II system for LI-events from new convective storms. Track ongoing lightning using Lightning Mapping Array (LMA) and pseudo-Geostationary Lightning Mapper (GLM) data to assess per-storm lightning trends (e.g., as tied to lightning jumps) and outline threat regions. Evaluate the ability to produce LI nowcasts through a "lightning threat" product, and obtain feedback from National Weather Service forecasters on its value as a decision support tool.

remote sensing↗

NASA SPoRT GOES-R Proving Ground Activities

The NASA Short-term Prediction Research and Transition (SPoRT) program is a partner with the GOES-R Proving Ground (PG) helping prepare forecasters understand the unique products to come from the GOES-R instrument suite. SPoRT is working collaboratively with other members of the GOES-R PG team and Algorithm Working Group (AWG) scientists to develop and disseminate a suite of proxy products that address specific forecast problems for the WFOs, Regional and National Support Centers, and other NOAA users. These products draw on SPoRT s expertise with the transition and evaluation of products into operations from the MODIS instrument and the North Alabama Lightning Mapping Array (NALMA). The MODIS instrument serves as an excellent proxy for the Advanced Baseline Imager (ABI) that will be aboard GOES-R. SPoRT has transitioned and evaluated several multi-channel MODIS products. The true and false color products are being used in natural hazard detection by several SPoRT partners to provide better observation of land features, such as fires, smoke plumes, and snow cover. Additionally, many of SPoRT s partners are coastal offices and already benefit from the MODIS sea surface temperature composite. This, along with other surface feature observations will be developed into ABI proxy products for diagnostic use in the forecast process as well as assimilation into forecast models. In addition to the MODIS instrument, the NALMA has proven very valuable to WFOs with access to these total lightning data. These data provide situational awareness and enhanced warning decision making to improve lead times for severe thunderstorm and tornado warnings. One effort by SPoRT scientists includes a lightning threat product to create short-term model forecasts of lightning activity. Additionally, SPoRT is working with the AWG to create GLM proxy data from several of the ground based total lightning networks, such as the NALMA. The evaluation will focus on the vastly improved spatial coverage of the GLM, but with the trade-off of lower resolution compared to the NALMA. In addition to the above tasks, SPoRT will make these data available in the NWS next generation display software, AWIPS II. This has already been successfully completed for the two basic GLM proxies. SPoRT will use these products to train forecasters on the capabilities of GOES-R and foster feedback to develop additional products, visualizations, and requirements beneficial to end users needs. These developments and feedback will be made available to the GOES-R Proving Ground for the upcoming 2010 Spring Program in Norman, Oklahoma.

Stano, Geoffrey T.↗

The Processing of Airspace Concept Evaluations Using FASTE-CNS as a Pre- or Post-Simulation CNS Analysis Tool

As NASA speculates on and explores the future of aviation, the technological and physical aspects of our environment increasing become hurdles that must be overcome for success. Research into methods for overcoming some of these selected hurdles have been purposed by several NASA research partners as concepts. The task of establishing a common evaluation environment was placed on NASA's Virtual Airspace Simulation Technologies (VAST) project (sub-project of VAMS), and they responded with the development of the Airspace Concept Evaluation System (ACES). As one examines the ACES environment from a communication, navigation or surveillance (CNS) perspective, the simulation parameters are built with assumed perfection in the transactions associated with CNS. To truly evaluate these concepts in a realistic sense, the contributions/effects of CNS must be part of the ACES. NASA Glenn Research Center (GRC) has supported the Virtual Airspace Modeling and Simulation (VAMS) project through the continued development of CNS models and analysis capabilities which supports the ACES environment. NASA GRC initiated the development a communications traffic loading analysis tool, called the Future Aeronautical Sub-network Traffic Emulator for Communications, Navigation and Surveillance (FASTE-CNS), as part of this support. This tool allows for forecasting of communications load with the understanding that, there is no single, common source for loading models used to evaluate the existing and planned communications channels; and that, consensus and accuracy in the traffic load models is a very important input to the decisions being made on the acceptability of communication techniques used to fulfill the aeronautical requirements. Leveraging off the existing capabilities of the FASTE-CNS tool, GRC has called for FASTE-CNS to have the functionality to pre- and post-process the simulation runs of ACES to report on instances when traffic density, frequency congestion or aircraft spacing/distance violations have occurred. The integration of these functions require that the CNS models used to characterize these avionic system be of higher fidelity and better consistency then is present in FASTE-CNS system. This presentation will explore the capabilities of FASTE-CNS with renewed emphasis on the enhancements being added to perform these processing functions; the fidelity and reliability of CNS models necessary to make the enhancements work; and the benchmarking of FASTE-CNS results to improve confidence for the results of the new processing capabilities.

Mainger, Steve↗

Evaluation and use of TOVS retrievals at NMC

In a current analysis procedure at the National Meteorological Center (NMC), a first guess analysis (from the latest 6 or 12 hour forecast) is updated by new data. In the newest analysis procedures, which are based on optimum interpolation, the observational correction depends on the size of preassigned, expected observational errors versus the size of expected errors in the first guess forecast. In the case of Northern Hemisphere ocean temperatures, the latter are around 2 degrees. The infrared satellite retrievals have a similar size "error". If (as assumed) their errors are uncorrelated with the first guess error, they will improve the analysis. The larger errors of the microwave retrievals, however, mean that they will be given little weight in the analysis. An evaluation of the TIROS-N retrievals is currently underway at NMC to determine the impact of satellite derived data within operational analysis schemes, to isolate possible problems within current retrieval methods, and to offer possible solutions for these problems.

Phillips, N. A.↗

Revisiting the Ground Magnetic Field Perturbations Challenge: A Machine Learning Perspective

Forecasting ground magnetic field perturbations has been a long-standing goal of the space weather community. The availability of ground magnetic field data and its potential to be used in geomagnetically induced current studies, such as risk assessment, have resulted in several forecasting efforts over the past few decades. One particular community effort was the Geospace Environment Modeling (GEM) challenge of ground magnetic field perturbations that evaluated the predictive capacity of several empirical and first principles models at both mid- and high-latitudes in order to choose an operative model. In this work, we use three different deep learning models-a feed-forward neural network, a long short-term memory recurrent network and a convolutional neural network-to forecast the horizontal component of the ground magnetic field rate of change (dB H /dt) over 6 different ground magnetometer stations and to compare as directly as possible with the original GEM challenge. We find that, in general, the models are able to perform at similar levels to those obtained in the original challenge, although the performance depends heavily on the particular storm being evaluated. We then discuss the limitations of such a comparison on the basis that the original challenge was not designed with machine learning algorithms in mind.

Victor A. Pinto↗

An evaluation of observations from satellites for the study and prediction of mesoscale events and cyclone events

Quantitative observational guidelines have been developed for the study and forecasting of mesoscale events and cyclone events; many of these guidelines have resulted from major meteorological field programs. Increasingly sophisticated satellite-borne instruments in low and geosynchronous orbits have provided valuable measurements of these events. The major deficiencies in the measurements taken from geosynchronous orbit today, relative to guidelines, are (1) the lack of temperature profiles and moisture profiles below clouds, and the poor vertical resolution of these profiles; (2) the insufficient combination of spatial resolution and temporal resolution and spectral intervals available in generating images (imaging); and (3) the lack of accurate precipitation mapping. Considerably more-powerful instrumentation is possible on geosynchronous satellites that can substantially reduce these deficiencies. The capability of advanced geosynchronous observations and those expected with future instrumentation in low-orbit are evaluated with respect to tropical cyclone and severe local-storm observational guidelines. A high percentage of the guidelines are expected to be fulfilled with geosynchronous measurements: (1) using microwave instruments for imaging and for obtaining temperature profiles and moisture profiles, (2) very high spectral-resolution infrared profiling, (3) very high spatial-resolution and very high temporal-resolution imaging, and (4) ozone mapping.

Shenk, William E.↗

Aircraft icing: Introduction

The objectives of the Workshop were as follows: (1) to assess the current understanding of fixed wing and rotorcraft operational icing environments and problems (2) to evaluate facilities requirements for R&D and certification purposes (3) to examine means of improving icing forecasts (4) to identify shortcomings in aeronautical icing knowledge which can be alleviated by new research and instrumentation development.

Enders, J. H.↗

Centralized Storm Information System (CSIS)

A final progress report is presented on the Centralized Storm Information System (CSIS). The primary purpose of the CSIS is to demonstrate and evaluate real time interactive computerized data collection, interpretation and display techniques as applied to severe weather forecasting. CSIS objectives pertaining to improved severe storm forecasting and warning systems are outlined. The positive impact that CSIS has had on the National Severe Storms Forecast Center (NSSFC) is discussed. The benefits of interactive processing systems on the forecasting ability of the NSSFC are described.

Norton, C. C.↗

Objective Interpolation of Scatterometer Winds

Global wind fields are produced by successive corrections that use measurements by the European Remote Sensing Satellite (ERS-1) scatterometer. The methodology is described. The wind fields at 10-meter height provided by the European Center for Medium-Range Weather Forecasting (ECMWF) are used to initialize the interpolation process. The interpolated wind field product ERSI is evaluated in terms of its improvement over the initial guess field (ECMWF) and the bin-averaged ERS-1 wind field (ERSB). Spatial and temporal differences between ERSI, ECMWF and ERSB are presented and discussed.

Tang, Wenquing↗

Ranger's Legacy

With its Landsat satellites, development of sensors, and advancement of processing techniques, NASA provided the initial technology base for another Earth-benefit application of image processing, Earth resources survey by means of remote sensing. Since each object has its own unique "signature," it is possible to distinguish among surface features and to generate computer-processed imagery identifying specific features of importance to resource managers. This capability, commercialized by Perceptive Scientific Instruments, Inc., offers practical application in such areas as agricultural crop forecasting, rangeland and forest management, land use planning, mineral and petroleum exploration, map making, water quality evaluation and disaster assessment. Major users of the technology have been federal, state, and local governments, but it is making its way into commercial operations, for example, resource exploration companies looking for oil, gas and mineral sources, and timber production firms seeking more efficient treeland management. Supporting both government and private users is a small industry composed of companies producing the processing hardware software. As is the case in the medical application, many of these companies are direct offspring of NASA's work.

Source record↗

Development of an OSSE Framework for a Global Atmospheric Data Assimilation System

Observing system simulation experiments (OSSEs) are powerful tools for estimating the usefulness of various configurations of envisioned observing systems and data assimilation techniques. Their utility stems from their being conducted in an entirely simulated context, utilizing simulated observations having simulated errors and drawn from a simulation of the earth's environment. Observations are generated by applying physically based algorithms to the simulated state, such as performed during data assimilation or using other appropriate algorithms. Adding realistic instrument plus representativeness errors, including their biases and correlations, can be critical for obtaining realistic assessments of the impact of a proposed observing system or analysis technique. If estimates of the expected accuracy of proposed observations are realistic, then the OSSE can be also used to learn how best to utilize the new information, accelerating its transition to operations once the real data are available. As with any inferences from simulations, however, it is first imperative that some baseline OSSEs are performed and well validated against corresponding results obtained with a real observing system. This talk provides an overview of, and highlights critical issues related to, the development of an OSSE framework for the tropospheric weather prediction component of the NASA GEOS-5 global atmospheric data assimilation system. The framework includes all existing observations having significant impact on short-term forecast skill. Its validity has been carefully assessed using a range of metrics that can be evaluated in both the OSSE and real contexts, including adjoint-based estimates of observation impact. A preliminary application to the Aeolus Doppler wind lidar mission, scheduled for launch by the European Space Agency in 2014, has also been investigated.

Gelaro, Ronald↗

Aggregation Tool to Create Curated Data albums to Support Disaster Recovery and Response

Despite advances in science and technology of prediction and simulation of natural hazards, losses incurred due to natural disasters keep growing every year. Natural disasters cause more economic losses as compared to anthropogenic disasters. Economic losses due to natural hazards are estimated to be around $6-$10 billion dollars annually for the U.S. and this number keeps increasing every year. This increase has been attributed to population growth and migration to more hazard prone locations such as coasts. As this trend continues, in concert with shifts in weather patterns caused by climate change, it is anticipated that losses associated with natural disasters will keep growing substantially. One of challenges disaster response and recovery analysts face is to quickly find, access and utilize a vast variety of relevant geospatial data collected by different federal agencies such as DoD, NASA, NOAA, EPA, USGS etc. Some examples of these data sets include high spatio-temporal resolution multi/hyperspectral satellite imagery, model prediction outputs from weather models, latest radar scans, measurements from an array of sensor networks such as Integrated Ocean Observing System etc. More often analysts may be familiar with limited, but specific datasets and are often unaware of or unfamiliar with a large quantity of other useful resources. Finding airborne or satellite data useful to a natural disaster event often requires a time consuming search through web pages and data archives. Additional information related to damages, deaths, and injuries requires extensive online searches for news reports and official report summaries. An analyst must also sift through vast amounts of potentially useful digital information captured by the general public such as geo-tagged photos, videos and real time damage updates within twitter feeds. Collecting and aggregating these information fragments can provide useful information in assessing damage in real time and help direct recovery efforts. The search process for the analyst could be made much more efficient and productive if a tool could go beyond a typical search engine and provide not just links to web sites but actual links to specific data relevant to the natural disaster, parse unstructured reports for useful information nuggets, as well as gather other related reports, summaries, news stories, and images. This presentation will describe a semantic aggregation tool developed to address similar problem for Earth Science researchers. This tool provides automated curation, and creates "Data Albums" to support case studies. The generated "Data Albums" are compiled collections of information related to a specific science topic or event, containing links to relevant data files (granules) from different instruments; tools and services for visualization and analysis; information about the event contained in news reports, and images or videos to supplement research analysis. An ontology-based relevancy-ranking algorithm drives the curation of relevant data sets for a given event. This tool is now being used to generate a catalog of Hurricane Case Studies at Global Hydrology Resource Center (GHRC), one of NASA's Distribute Active Archive Centers. Another instance of the Data Albums tool is currently being created in collaboration with NASA/MSFC's SPoRT Center, which conducts research on unique NASA products and capabilities that can be transitioned to the operational community to solve forecast problems. This new instance focuses on severe weather to support SPoRT researchers in their model evaluation studies

Ramachandran, Rahul↗

Improved Peatland Hydrology and L-Band Microwave Radiative Transfer Modeling in Version 7 of the SMAP Level-4 Soil Moisture Data Assimilation Product

The NASA Soil Moisture Active Passive (SMAP) mission Level-4 Soil Moisture (L4_SM) product provides global, 9-km resolution, 3-hourly surface (0-5 cm) and root-zone (0-100 cm) soil moisture from April 2015 to present with a mean latency of 2.5 days from the time of observation. The L4_SM estimates are derived from the assimilation of SMAP L-band (1.4 GHz) brightness temperature (Tb) observations into the NASA Catchment land surface model. The recently released Version 7 of L4_SM comprises two key advances. First, the Catchment model now includes the recently developed PEATCLSM hydrology module for peatlands and uses an updated global map of peatlands. Second, revised parameters are used in the L-band radiative transfer model that converts the simulated soil moisture and temperature estimates into Tb predictions for use in the radiance-based L4_SM analysis. Specifically, the L-band scattering albedo, soil roughness, and vegetation opacity climatology are obtained from the SMAP Level-2 dual-channel soil moisture retrieval product. In this presentation, we examine the improvements gained with the model and parameter changes. In peatlands, the dynamics of water table depth, surface soil moisture and evapotranspiration are considerably improved in L4_SM Version 7 over Version 6 when evaluated against in situ measurements. These improvements are also manifested in smaller Tb observation-minus-forecast residuals.

SMAP↗

Global NWP Impacts of Infrared Sounders from Geostationary Orbit

The Geostationary eXtended Observations (GeoXO) program, expected to launch in the 2030s, includes a proposed infrared (IR) sounder that would provide persistent atmospheric profile observations over much of the western hemisphere. In preparation, the National Aeronautics and Space Administration (NASA) Global Modeling and Assimilation Office (GMAO) observing system simulation experiment (OSSE) framework was used to assess the impact of such an instrument on global numerical weather prediction (NWP) as part of a global “ring” of such instruments. Building on previous preliminary studies, an evaluation of the impact of geostationary IR sounders will be presented with foci on the analysis, forecasts, and the forecast sensitivity observation impact (FSOI) metric, including an examination of the consequences for tropical cyclone representation. Overall, assimilation of geostationary IR provide a beneficial impact for NWP applications.

Erica L. Mcgrath-spangler↗

Evaluating Observation Influence on Regional Water Budgets in Reanalyses

The assimilation of observations in reanalyses incurs the potential for the physical terms of budgets to be balanced by a term relating the fit of the observations relative to a forecast first guess analysis. This may indicate a limitation in the physical processes of the background model, or perhaps inconsistencies in the observing system and its assimilation. In the MERRA reanalysis, an area of long term moisture flux divergence over land has been identified over the Central United States. Here, we evaluate the water vapor budget in this region, taking advantage of two unique features of the MERRA diagnostic output; 1) a closed water budget that includes the analysis increment and 2) a gridded diagnostic output data set of the assimilated observations and their innovations (e.g. forecast departures). In the Central United States, an anomaly occurs where the analysis adds water to the region, while precipitation decreases and moisture flux divergence increases. This is related more to a change in the observing system than to a deficiency in the model physical processes. MERRAs Gridded Innovations and Observations (GIO) data narrow the observations that influence this feature to the ATOVS and Aqua satellites during the 06Z and 18Z analysis cycles. Observing system experiments further narrow the instruments that affect the anomalous feature to AMSUA (mainly window channels) and AIRS. This effort also shows the complexities of the observing system, and the reactions of the regional water budgets in reanalyses to the assimilated observations.

Atmospheric Reanalysis↗

Characterization of Transport Errors in Chemical Forecasts from a Global Tropospheric Chemical Transport Model

We propose a new methodology to characterize errors in the representation of transport processes in chemical transport models. We constrain the evaluation of a global three-dimensional chemical transport model (GEOS-CHEM) with an extended dataset of carbon monoxide (CO) concentrations obtained during the Transport and Chemical Evolution over the Pacific (TRACE-P) aircraft campaign. The TRACEP mission took place over the western Pacific, a region frequently impacted by continental outflow associated with different synoptic-scale weather systems (such as cold fronts) and deep convection, and thus provides a valuable dataset. for our analysis. Model simulations using both forecast and assimilated meteorology are examined. Background CO concentrations are computed as a function of latitude and altitude and subsequently subtracted from both the observed and the model datasets to focus on the ability of the model to simulate variability on a synoptic scale. Different sampling strategies (i.e., spatial displacement and smoothing) are applied along the flight tracks to search for systematic model biases. Statistical quantities such as correlation coefficient and centered root-mean-square difference are computed between the simulated and the observed fields and are further inter-compared using Taylor diagrams. We find no systematic bias in the model for the TRACE-P region when we consider the entire dataset (i.e., from the surface to 12 km ). This result indicates that the transport error in our model is globally unbiased, which has important implications for using the model to conduct inverse modeling studies. Using the First-Look assimilated meteorology only provides little improvement of the correlation, in comparison with the forecast meteorology. These general statements can be refined when the entire dataset is divided into different vertical domains, i.e., the lower troposphere (less than 2 km), the middle troposphere (2-6 km), and the upper troposphere (greater than 6 km). The best agreement between the observations and the model is found in the lower and middle troposphere. Downward displacements in the lower troposphere provide a better fit with the observed value, which could indicate a problem in the representation of boundary layer height in the model. Significant improvement is also found for downward and southward displacements in the upper troposphere. There are several potential sources of errors in our simulation of the continental outflow in the upper troposphere which could lead to such biases, including the location and/or the strength of deep convective cells as well as that of wildfires in Southeast Asia.

Bey, I.↗