SEARCH · Search NASA

Results for “forecast evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

All sources NASA NTRS DOE OSTI

NASA NTRS2002

Recent Observing System Simulation Experiments at the NASA DAO

Since the advent of meteorological satellites in the 1960's, numerous experiments have been conducted in order t o evaluate the impact of these and other data on atmospheric analysis and prediction. Such studies have included both OSE'S (Observing System Experiments) and OSSE's (Observing System Simulation Experiments). The OSE's were conducted to evaluate the impact of specific observations or classes of observations on analyses and forecasts. Such experiments have been performed for selected types of conventional data and for various satellite data sets as they became avail- able. (See for example the 1989 ECMWF/EUMETSAT workshop proceedings on "The use of satellite data in operational numerical weather prediction" and the references contained therein.) The OSSE's were con- ducted to evaluate the potential for future observing systems to improve Numerical Weather Prediction (NWP) and to plan for the Global Weather Experiment and more recently for EOS. In addition, OSSE's have been run t o evaluate trade-offs in the design of observing systems and observing net-works, and to test new methodology for data assimilation.

Atlas, Robert↗

NASA NTRS2002

Requirements for Space-Based Wind Lidar

Global wind profiles are needed for a wide range of meteorological applications. Since the 1980's, observing system simulation experiments have been conducted in order to evaluate the potential impact of space-based wind profiler data on numerical weather prediction, and to evaluate trade-offs in lidar design. These experiments indicated tremendous potential for satellite lidar observations to improve atmospheric analyses and forecasts. More recent experiments are aimed at assessing the precise requirements for space-based lidar wind profile data and to evaluate the potential for alternative technologies. At the workshop, OSSE methodology, and results from experiments conducted at the DAO to the define requirements for space-based lidar wind will be presented.

Atlas, Robert M.↗

NASA NTRS2002

TIES for Dummies 3rd Edition (Technology Identification, Evaluation, and Selection) Basic how to's to implement the TIES method

The TIES method is a forecasting environment whereby the decision-maker has the ability to easily assess and trade-off the impact of various technologies without sophisticated and time-consuming mathematical formulations. TIES provides a methodical approach where technically feasible alternatives can be identified with accuracy and speed to reduce design cycle time, and subsequently, life cycle costs, and was achieved through the use of various probabilistic methods, such as Response Surface Methodology and Monte Carlo Simulations. Furthermore, structured and systematic techniques are utilized from other fields to identify possible concepts and evaluation criteria by which comparisons can be made. This objective is achieved by employing the use of Morphological Matrices and Multi-Attribute Decision Making techniques. Through the execution of each step, a family of design alternatives for a given set of customer requirements can be identified and assessed subjectively or objectively. This methodology allows for more information (knowledge) to be brought into the earlier phases of the design process and will have direct implications on the affordability of the system. The increased knowledge allows for optimum allocation of company resources and quantitative justification for program decisions. Finally, the TIES method provided novel results and quantitative justification to facilitate decision making in the early stages of design so as to produce affordable and quality products.

Kirby, Michelle R.↗

NASA NTRS2020

Assessment of Natural and Anthropogenic Aerosol Air Pollution In the Middle East Using MERRA-2, CAMS Data Assimilation Products, and High-Resolution WRF-Chem Model Simulations

Modern-Era Retrospective analysis for Research and Applications v.2 (MERRA-2), Copernicus Atmosphere Monitoring Service Operational Analysis (CAMS-OA), and a high-resolution regional Weather Research and Forecasting model coupled with chemistry (WRF-Chem) were used to evaluate natural and anthropogenic particulate matter (PM) air pollution in the Middle East (ME) during 2015–2016. Two Moderate Resolution Imaging Spectrometer (MODIS) retrievals – combined product Deep Blue and Deep Target (MODIS-DB&DT) and Multi-Angle Implementation of Atmospheric Correction (MAIAC) – and Aerosol Robotic Network (AERONET) aerosol optical depth (AOD) observations as well as in situ PM measurements for 2016 were used for validation of the WRF-Chem output and both assimilation products. MERRA-2 and CAMS-OA assimilate AOD observations. WRF-Chem is a free-running model, but dust emission in WRF-Chem is tuned to fit AOD and aerosol volume size distributions obtained from AERONET. MERRA-2 was used to construct WRF-Chem initial and boundary conditions both for meteorology and chemical and aerosol species. SO2 emissions in WRF-Chem are based on the novel OMI-HTAP SO2 emission dataset. The correlation with the AERONET AOD is highest for MERRA-2 (0.72–0.91), MAIAC (0.63–0.96), and CAMSOA (0.65–0.87), followed by MODIS-DB&DT (0.56–0.84) and WRF-Chem (0.43–0.85). However, CAMS-OA has a relatively high positive mean bias with respect to AERONET AOD. The spatial distributions of seasonally averaged AODs from WRF-Chem, assimilation products, and MAIAC are well correlated with MODIS-DB&DT AOD product. MAIAC has the highest correlation (R = 0.8), followed by MERRA-2 (R = 0.66), CAMS-OA (R = 0.65), and WRF-Chem (R = 0.61). WRF-Chem, MERRA-2, and MAIAC underestimate and CAMS-OA overestimates MODIS-DB&DT AOD. The simulated and observed PM concentrations might differ by a factor of 2 because it is more challenging for the model and the assimilation products to reproduce PM concentration measured within the city. Although aerosol fields in WRF-Chem and assimilation products are entirely consistent, WRF-Chem is preferable for analysis of regional air quality over the ME due to its higher spatial resolution and better SO2 emissions. The WRF-Chem’s PM background concentrations exceed the World Health Organization (WHO) guidelines over the entire ME. Mineral dust is the major contributor to PM (≈ 75%–95%) compared to other aerosol types. Near and downwind from the SO2 emission sources, non-dust aerosols (primarily sulfate) contribute up to 30% to PM(sub 2.5). The contribution of sea salt to PM in coastal regions can reach 5%. The contributions of organic matter, black carbon and organic carbon to PM over the Middle East are insignificant. In the major cities over the Arabian Peninsula, the 90th percentile of PM(sub 10) and PM(sub 2.5) (particles with diameters less than 10 and 2.5 μm, respectively) daily mean surface concentrations exceed the corresponding Kingdom of Saudi Arabia air quality limits. The contribution of the non-dust component to PM(sub 2.5) is < 25%, which limits the emission control effect on air quality. The mitigation of the dust effect on air quality requires the development of environment-based approaches like growing tree belts around the cities and enhancing in-city vegetation cover. The WRF-Chem configuration presented in this study could be a prototype of a future air quality forecast system that warns the population against air pollution hazards.

Alexander Ukhov↗

NASA NTRS2022

Observation Impact and Information Retention in the Lower Troposphere of the GMAO GEOS Data Assimilation System

In this study, we have assessed the effectiveness of the use of existing observing systems in the lower troposphere in the GEOS hybrid–4DEnVar data assimilation system through a set of observing system experiments. The results show that microwave radiances have a large impact in the Southern Hemisphere and Tropical ocean, but the large influence is mostly observed above 925 hPa and dissipates relatively quickly with longer forecast lead times. Conventional data information holds better in the forecast ranging from the surface to 100 hPa, depending on the field evaluated, in the Northern Hemisphere and lowest model levels in the Tropics. Infrared radiances collectively have much less impact in the lower troposphere. Removing surface observations has small but persistent impact on specific humidity in the upper atmosphere, but small or negligible impact on planetary boundary layer (PBL) height and temperature. The model responses to the incremental analysis update (IAU) forcing are also analyzed. In the IAU assimilation window, the physics responds strongly to the IAU forcing in the lower troposphere, and the changes of physics tendency in the lower troposphere and hydrodynamics tendency in the mid- and upper troposphere are viewed as beneficial to the reduction of state error covariance. In the subsequent forecast, the model tendencies continue to deviate further from the original free forecast with forecast lead times around 300–400 hPa, but physics tendency has showed signs of returning to its original free forecast mechanisms at 1-day forecast in the lower troposphere.

GEOS↗

NASA NTRS2014

Prediction of the Arctic Oscillation in Boreal Winter by Dynamical Seasonal Forecasting Systems

This study assesses the skill of boreal winter Arctic Oscillation (AO) predictions with state-of-the-art dynamical ensemble prediction systems (EPSs): GloSea4, CFSv2, GEOS-5, CanCM3, CanCM4, and CM2.1. Long-term reforecasts with the EPSs are used to evaluate how well they represent the AO and to assess the skill of both deterministic and probabilistic forecasts of the AO. The reforecasts reproduce the observed changes in the large-scale patterns of the Northern Hemispheric surface temperature, upper level wind, and precipitation associated with the different phases of the AO. The results demonstrate that most EPSs improve upon persistence skill scores for lead times up to 2 months in boreal winter, suggesting some potential for skillful prediction of the AO and its associated climate anomalies at seasonal time scales. It is also found that the skill of AO forecasts during the recent period (1997-2010) is higher than that of the earlier period (1983-1996).

Arctic Oscillation↗

NASA NTRSDate not supplied

NASA’s Simulation Activities for Evaluating UAM Concept of Operations

NASA’s vision for Advanced Air Mobility (AAM) is to help emerging aviation markets to safely develop an air transportation system that moves people and cargo between places previously not served or underserved by aviation using revolutionary new aircraft types. Urban Air Mobility (UAM), the concept of expanding transportation networks by serving short flights to transport people and goods around metropolitan areas, is part of a larger paradigm shift toward AAM, in which new technologies and business models are enabling transformational applications of aviation. In NASA’s Concept of Operations for UAM, Providers of Services for UAM (PSU) play a key role to enable safe and efficient UAM operations by sharing operational data among UAM operators. PSUs provide a pre-departure strategic conflict management service that establishes operational plans for new flights and coordinates them with all relevant operations in shared airspace. The strategic conflict management aims to minimize the need for tactical separation provision, while considering anticipated traffic demand, vertiport capacity and availability, forecasted weather, and airspace restrictions. NASA is currently preparing a new lab evaluation activity with aviation industry partners, called the X4 simulation, to demonstrate their airspace services and capabilities and test new information exchange requirements for UAM operations. This simulation activity will support NASA’s National Campaign (NC) flight tests which are planned over the next several years to guide the collective community and stakeholders through a series of scenario-based test activities that involve vehicles and airspace management services operating in a live test environment. This presentation will provide an overview of the X4 simulation, including objectives, scenarios, assumptions, system architecture, and test schedule. The X4 simulation will evaluate the interconnectivity and operational intent sharing between PSUs for multiple operators, as well as the performance of strategic conflict management.

Urban Air Mobility, Simulation, Advanced Air Mobil↗

NASA NTRSDate not supplied

Optimization of Airport Runway Configuration with Forecast-Augmented Offline Reinforcement Learning

Runway configuration Management (RCM) governs the optimal utilization of runways based on variables such as traffic and meteorological conditions, making it a daunting task in air traffic management due to its dependency on volatile operational and environmental factors. This paper improves upon our previous work [1] on using offline model-free reinforcement learning for creating a Runway Configuration Assistance (RCA) decision-support tool. A novel integration of forecast data from LAMP (Localized Aviation Model Output Statistics Program) and TAF (Terminal Area Forecast) is introduced, enhancing the tool’s accuracy and also its adaptability to quick wind changes. The performance is evaluated using two major US airports, Charlotte Douglas International Airport (CLT) and Denver International Airport (DEN). To counter scalability issues presented by the addition of discrete forecast variables, we transitioned to a continuous state space model, ensuring scalability and inclusion of longer forecast data. The results of our experiments reflect significant improvements in the RCA tool’s prediction accuracy.

Sumanth Nethi↗

NASA NTRS1999

Improving Reanalyses Using TRMM and SSM/I-Derived Precipitation and Total Precipitable Water Observations

Global reanalyses currently contain significant errors in the primary fields of the hydrological cycle such as precipitation, evaporation, moisture, and the related cloud fields, especially in the tropics. The Data Assimilation Office (DAO) at the NASA Goddard Space Flight Center has been exploring the use of rainfall and total precipitable water (TPW) observations from the Tropical Rainfall Measuring Mission (TRMM) Microwave Imager (TMI) and the Special Sensor Microwave/ Imager (SSM/I) instruments to improve these fields in reanalyses. The DAO has developed a "1+1"D procedure to assimilate 6-hr averaged rainfall and TPW into the Goddard Earth Observing System (GEOS) Data Assimilation System (DAS). The algorithm is based on a 6-hr time integration of a column version of the GEOS DAS. The "1+1" designation refers to one spatial dimension plus one temporal dimension. The scheme minimizes the least-square differences between the satellite-retrieved rain rates and those produced by the column model over the 6-hr analysis window. The control variables are analysis increments of moisture within the Incremental Analysis Update (IAU) framework of the GEOS DAS. This 1+1D scheme, in its generalization to four dimensions, is related to the standard 4D variational assimilation but differs in its choice of the control variable. Instead of estimating the initial condition at the beginning of the assimilation cycle, it estimates the constant IAU forcing applied over a 6-hr assimilation cycle. In doing so, it imposes the forecast model as a weak constraint in a manner similar to the variational continuous assimilation techniques. We present results from an experiment in which the observed rain rate and TPW are assumed to be "perfect". They show that assimilating the TMI and SSM/I-derived surface precipitation and TPW observations improves not only the precipitation and moisture fields but also key climate parameters directly linked to convective activities such as clouds, the outgoing longwave radiation, and the large-scale circulation in the tropics. In particular, assimilating these data types reduce the state-dependent systematic errors in the assimilated products. The improved analysis also leads to a better short-range forecast, but the impact is modest compared with improvements in the time-averaged fields. These results suggest that, in the presence of biases and other errors of the forecast model, it is possible to improve the time-averaged "climate content" in the assimilated data without comparable improvements in the short-range forecast skill. Results of this experiment provide a useful benchmark for evaluating error covariance models for optimal use of these data types.

Hou, Arthur Y.↗

NASA NTRS2005

Analysis of Multiple Precipitation Products and Preliminary Assessment of Their Impact on Global Land Data Assimilation System (GLDAS) Land Surface States

Land surface models (LSMs) are computer programs, similar to weather and climate prediction models, which simulate the stocks and fluxes of water (including soil moisture, snow, evaporation, and runoff) and energy (including the temperature of and sensible heat released from the soil) after they arrive on the land surface as precipitation and sunlight. It is not currently possible to measure all of the variables of interest everywhere on Earth with sufficient accuracy and space-time resolution. Hence LSMs have been developed to integrate the available observations with our understanding of the physical processes involved, using powerful computers, in order to map these stocks and fluxes as they change in time. The maps are used to improve weather forecasts, support water resources and agricultural applications, and study the Earth's water cycle and climate variability. NASA's Global Land Data Assimilation System (GLDAS) project facilitates testing of several different LSMs with a variety of input datasets (e.g., precipitation, plant type). Precipitation is arguably the most important input to LSMs. Many precipitation datasets have been produced using satellite and rain gauge observations and weather forecast models. In this study, seven different global precipitation datasets were evaluated over the United States, where dense rain gauge networks contribute to reliable precipitation maps. We then used the seven datasets as inputs to GLDAS simulations, so that we could diagnose their impacts on output stocks and fluxes of water. In terms of totals, the Climate Prediction Center (CPC) Merged Analysis of Precipitation (CMAP) had the closest agreement with the US rain gauge dataset for all seasons except winter. The CMAP precipitation was also the most closely correlated in time with the rain gauge data during spring, fall, and winter, while the satellitebased estimates performed best in summer. The GLDAS simulations revealed that modeled soil moisture is highly sensitive to precipitation, with differences in spring and summer as large as 45% depending on the choice of precipitation input.

Gottschalck, Jon↗

NASA NTRS2011

Validation of Space Weather Models at Community Coordinated Modeling Center

The Community Coordinated Modeling Center (CCMC) is a multiagency partnership, which aims at the creation of next generation space weather modes. CCMC goal is to support the research and developmental work necessary to substantially increase space weather modeling capabilities and to facilitate advanced models deployment in forecasting operations. The CCMC conducts unbiased model testing and validation and evaluates model readiness for operational environment. The presentation will demonstrate the recent progress in CCMC metrics and validation activities.

Kuznetsova, M. M.↗

NASA NTRS2012

Using Historical Precipitation, Temperature, and Runoff Observations to Evaluate Evaporation Formulations in Land Surface Models

Key to translating soil moisture memory into subseasonal precipitation and air temperature forecast skill is a realistic treatment of evaporation in the forecast system used - in particular, a realistic treatment of how evaporation responds to variations in soil moisture. The inherent soil moisture-evaporation relationships used in today's land surface models (LSMs), however, arguably reflect little more than guesswork given the lack of evaporation and soil moisture data at the spatial scales represented by regional and global models. Here we present a new approach for evaluating this critical aspect of LSMs. Seasonally averaged precipitation is used as a proxy for seasonally-averaged soil moisture, and seasonally-averaged air temperature is used as a proxy for seasonally-averaged evaporation (e.g., more evaporative cooling leads to cooler temperatures)~ the relationship between historical precipitation and temperature measurements accordingly mimics in certain important ways nature's relationship between soil moisture and evaporation. Additional information on the relationship is gleaned from joint analysis of precipitation and streamflow measurements. An experimental framework that utilizes these ideas to guide the development of an improved soil moisture-evaporation relationship is described and demonstrated.

Koster, Randal D.↗

NASA NTRS1980

Global crop forecasting

The needs for and remote sensing means of global crop forecasting are discussed, and key results of the Large Area Crop Inventory Experiment (LACIE) are presented. Current crop production estimates provided by foreign countries are shown often to be inadequate, and the basic elements of crop production forecasts are reviewed. The LACIE project is introduced as a proof-of-concept experiment designed to assimilate remote sensing technology, monitor global wheat production, evaluate key technical problems, modify the technique accordingly and demonstrate the feasibility of a global agricultural monitoring system. The global meteorological data, sampling and aggregation techniques, Landsat data analysis procedures and yield forecast procedures used in the experiment are outlined. Accuracy assessment procedures employed to evaluate LACIE technology performance are presented, and improvements in system efficiency and capacity during the three years of operation are pointed out. Results of LACIE estimates of Soviet, U.S. and Canadian wheat production are presented which demonstrate the feasibility and accuracy of the remote-sensing approach for global food and fiber monitoring.

Macdonald, R. B.↗

NASA NTRS2017

Meteoroid Environment Modeling: the Meteoroid Engineering Model and Shower Forecasting

The meteoroid environment is often divided conceptually into meteor showers plus a sporadic background component. The sporadic complex poses the bulk of the risk to spacecraft, but showers can produce significant short-term enhancements of the meteoroid flux. The Meteoroid Environment Office (MEO) has produced two environment models to handle these cases: the Meteoroid Engineering Model (MEM) and an annual meteor shower forecast. Both MEM and the forecast are used by multiple manned spaceflight projects in their meteoroid risk evaluation, and both tools are being revised to incorporate recent meteor velocity, density, and timing measurements. MEM describes the sporadic meteoroid complex and calculates the flux, speed, and directionality of the meteoroid environment relative to a user-supplied spacecraft trajectory, taking the spacecraft's motion into account. MEM is valid in the inner solar system and offers near-Earth and cis-lunar environments. While the current version of MEM offers a nominal meteoroid environment corresponding to a single meteoroid bulk density, the next version of MEMR3 will offer both flux uncertainties and a density distribution in addition to a revised near-Earth environment. We have updated the near-Earth meteor speed distribution and have made the first determination of uncertainty in this distribution. We have also derived a meteor density distribution from the work of Kikwaya et al. (2011). The annual meteor shower forecast takes the form of a report and data tables that can be used in conjunction with an existing MEM assessment. Fluxes are typically quoted to a constant limiting kinetic energy in order to comport with commonly used ballistic limit equations. For the 2017 annual forecast, the MEO substantially revised the list of showers and their characteristics using 14 years of meteor flux measurements from the Canadian Meteor Orbit Radar (CMOR). Defunct or insignificant showers were removed and the temporal profiles of many showers were improved. In 2016 the MEO also adapted the forecast to the cislunar environment for the first time. We plan to make additional improvements to the model in the next two years using optical meteor flux measurements and mass indices.

Moorhead, Althea V.↗

NASA NTRS2017

Meteoroid Environment Modeling: the Meteoroid Engineering Model and Shower Forecasting

Moorhead, Althea V.↗

NASA NTRS1983

An evaluation of the synoptic- and mesoscale predictability of the Mesoscale Atmospheric Simulation System (MASS 2.0) model

A report is presented regarding the synoptic- and mesoscale predictive capabilities of a regional-scale numerical weather prediction model known as the Mesoscale Atmospheric Simulation System (MASS, Version 2.0). The development of this model has been discussed by Kaplan et al. (1982). An evaluation of the performance of MASS 2.0 is based on the study of a sample of approximately thirty 12 h and 24 h forecasts of atmospheric flow patterns over the U.S. during spring and early summer of 1982. A description of model systems is provided, and synoptic-scale evaluation methods are considered along with aspects of mesoscale evaluation methodology, examples of coherent mesoscale information provided by MASS 2.0, the results of a diagnostic study of mesoscale convective systems (MCS), and the results of a limited real-time forecast experiment.

Koch, S. E.↗

NASA NTRS2007

Coupled Stochastic Time-Inverted Lagrangian Transport/Weather Forecast and Research/Vegetation Photosynthesis and Respiration Model: Simulations of Tower-Based and Airborne CO2 Measurements - Part II

This paper evaluates simulations of atmospheric CO2 measured in 2004 at continental surface and airborne receptors, intended to test the capability to use data with high temporal and spatial resolution for analyses of carbon sources and sinks at regional and continental scales. The simulations were performed using the Stochastic Time-Inverted Lagrangian Transport (STILT) model driven by the Weather Forecast and Research (WRF) model, and linked to surface fluxes from the satellite-driven Vegetation Photosynthesis and Respiration Model (VPRM). The simulations provide detailed representations of hourly CO2 tower data and reproduce the shapes of airborne vertical profiles with high fidelity. WRF meteorology gives superior model performance compared with standard meteorological products, and the impact of including WRF convective mass fluxes in the STILT trajectory calculations is significant in individual cases. Important biases in the simulation are associated with the nighttime CO2 build-up and subsequent morning transition to convective conditions, and with errors in the advected lateral boundary condition. Comparison of STILT simulations driven by the WRF model against those driven by the Brazilian variant of the Regional Atmospheric Modeling System (BRAMS) shows that model-to-model differences are smaller than between an individual transport model and observations, pointing to systematic errors in the simulated transport. Future developments in the WRF model s data assimilation capabilities, basic research into the fundamental aspects of trajectory calculations, and intercomparison studies involving other transport models, are possible venues for reducing these errors. Overall, the STILT/WRF/VPRM offers a powerful tool for continental and regional scale carbon flux estimates.

Eluszkiewicz, Janusz↗

NASA NTRS2009

Satellite Proving Ground for the GOES-R Geostationary Lightning Mapper (GLM)

The key mission of the Satellite Proving Ground is to demonstrate new satellite observing data, products and capabilities in the operational environment to be ready on Day 1 to use the GOES-R suite of measurements. Algorithms, tools, and techniques must be tested, validated, and assessed by end users for their utility before they are finalized and incorporated into forecast operations. The GOES-R Proving Ground for the Geostationary Lightning Mapper (GLM) focuses on evaluating how the infusion of the new technology, algorithms, decision aids, or tailored products integrate with other available tools (weather radar and ground strike networks; nowcasting systems, mesoscale analysis, and numerical weather prediction models) in the hands of the forecaster responsible for issuing forecasts and warning products. Additionally, the testing concept fosters operation and development staff interactions which will improve training materials and support documentation development. Real-time proxy total lightning data from regional VHF lightning mapping arrays (LMA) in Northern Alabama, Central Oklahoma, Cape Canaveral Florida, and the Washington, DC Greater Metropolitan Area are the cornerstone for the GLM Proving Ground. The proxy data will simulate the 8 km Event, Group and Flash data that will be generated by GLM. Tailored products such as total flash density at 1-2 minute intervals will be provided for display in AWIPS-2 to select NWS forecast offices and national centers such as the Storm Prediction Center. Additional temporal / spatial combinations are being investigated in coordination with operational needs and case-study proxy data and prototype visualizations may also be generated from the NASA heritage Lightning Imaging Sensor and Optical Transient Detector data. End users will provide feedback on the utility of products in their operational environment, identify use cases and spatial/temporal scales of interest, and provide feedback to the developers for adjusted or new products.

Goodman, Steven J.↗