Search NASA⌕ Search

SEARCH · Search NASA

Results for “model skill”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS1988

Numerical prediction experiments related to the summer 1980 U.S. heat wave

The GLA (Goddard Laboratory for Atmosphere) fourth-order model predictions of the summer 1980 heat wave are examined to assess the skill of the model in numerically forecasting the initiation of the heat wave and the sensitivity of the model predictions to initial conditions and boundary forcing. Results of experiments indicate that the model predictions are sensitive to the initial conditions. The sensitivity of the model to anomalous boundary forcing is found to be generally larger than its sensitivity to initial conditions at extended ranges. The results of the study suggest that models may be useful in forecasting similar events if adequate determinations of the evolving boundary forcing can be made in real time.

Atlas, R.↗
NASA NTRS2020

Prediction skill of the 2012 U.S. Great Plains flash drought in Subseasonal Experiment (SubX) models

Rapid-onset droughts, known as flash droughts, can have devastating impacts on agriculture, water resources, and ecosystems. The ability to predict flash droughts in advance would greatly enhance our preparation for them and potentially mitigate their impacts. Here, we investigate the prediction skill of the extreme 2012 flash drought over the U.S. Great Plains at subseasonal lead times (3 weeks or more in advance) in global forecast systems participating in the Subseasonal Experiment (SubX). An additional comprehensive set of subseasonal hindcasts with NASA’s GEOS model, a SubX model with relatively high prediction skill, was performed to investigate the separate contributions of atmospheric and land initial conditions to flash drought prediction skill. The results show that the prediction skill of the SubX models is quite variable. While skillful predictions are restricted to within the first two forecast weeks in most models, skill is considerably better (3-4 weeks or more) for certain models and initialization dates. The enhanced prediction skill is found to originate from two robust sources: 1) accurate soil moisture initialization once dry soil conditions are established, and 2) the satisfactory representation of quasi-stationary cross-Pacific Rossby wave trains that lead to the rapid intensification of flash droughts. Evidence is provided that the importance of soil moisture initialization applies more generally to central U.S. summer flash droughts. Our results corroborate earlier findings that accurate soil moisture initialization is important for skillful subseasonal forecasts and highlight the need for additional research on the sources and predictability of drought-inducing quasi-stationary atmospheric circulation anomalies.

Anthony M DeAngelis↗
NASA NTRS2007

Weather Research and Forecasting Model Sensitivity Comparisons for Warm Season Convective Initiation

Mesoscale weather conditions can significantly affect the space launch and landing operations at Kennedy Space Center (KSC) and Cape Canaveral Air Force Station (CCAFS). During the summer months, land-sea interactions that occur across KSC and CCAFS lead to the formation of a sea breeze, which can then spawn deep convection. These convective processes often last 60 minutes or less and pose a significant challenge to the forecasters at the National Weather Service (NWS) Spaceflight Meteorology Group (SMG). The main challenge is that a "GO" forecast for thunderstorms and precipitation at the Shuttle Landing Facility is required at the 90 minute deorbit decision for End Of Mission (EOM) and at the 30 minute Return To Launch Site (RTLS) decision. Convective initiation, timing, and mode also present a forecast challenge for the NWS in Melbourne, FL (MLB). The NWS MLB issues such tactical forecast information as Terminal Aerodrome Forecasts (TAF5), Spot Forecasts for fire weather and hazardous materials incident support, and severe/hazardous weather Watches, Warnings, and Advisories. Lastly, these forecasting challenges can also affect the 45th Weather Squadron (45 WS), which provides comprehensive weather forecasts for shuttle launch, as well as ground operations, at KSC and CCAFS. The need for accurate mesoscale model forecasts to aid in their decision making is crucial. This study specifically addresses the skill of different model configurations in forecasting warm season convective initiation. Numerous factors influence the development of convection over the Florida peninsula. These factors include sea breezes, river and lake breezes, the prevailing low-level flow, and convergent flow due to convex coastlines that enhance the sea breeze. The interaction of these processes produces the warm season convective patterns seen over the Florida peninsula. However, warm season convection remains one of the most poorly forecast meteorological parameters. To determine which configuration options are best to address this specific forecast concern, the Weather Research and Forecasting (WRF) model, which has two dynamical cores - the Advanced Research WRF (ARW) and the Non-hydrostatic Mesoscale Model (NMM) was employed. In addition to the two dynamical cores, there are also two options for a "hot-start" initialization of the WRF model - the Local Analysis and Prediction System (LAPS; McGinley 1995) and the Advanced Regional Prediction System (ARPS) Data Analysis System (ADAS; Brewster 1996). Both LAPS and ADAS are 3- dimensional weather analysis systems that integrate multiple meteorological data sources into one consistent analysis over the user's domain of interest. This allows mesoscale models to benefit from the addition of highresolution data sources. Having a series of initialization options and WRF cores, as well as many options within each core, provides SMG and MLB with considerable flexibility as well as challenges. It is the goal of this study to assess the different configurations available and to determine which configuration will best predict warm season convective initiation.

Watson, Leela R.↗
NASA NTRS1985

Synoptic scale forecast skill and systematic errors in the MASS 2.0 model

The synoptic scale performance characteristics of MASS 2.0 are determined by comparing filtered 12-24 hr model forecasts to same-case forecasts made by the National Meteorological Center's synoptic-scale Limited-area Fine Mesh model. Characteristics of the two systems are contrasted, and the analysis methodology used to determine statistical skill scores and systematic errors is described. The overall relative performance of the two models in the sample is documented, and important systematic errors uncovered are presented.

Koch, S. E.↗
NASA NTRS2018

Impacts of NASA's Earth Observations on Subseasonal and Seasonal Forecasts

NASA's suite of Earth-observing satellites provides a unique view of many processes on Earth, with relevance on timescales ranging from hours to weeks and even years. NASA's observations span all parts of the Earth system: atmospheric, ocean, land and cryosphere, and include physical, chemical and biological components. This presentation explores use of NASA observations in extended-range prediction, from many days to months, using the Goddard Earth Observing System (GEOS) assimilation and predictive modeling capabilities. The skill of a weather forecast is linked to the fidelity of the initialization process (data assimilation) and the realistic representation of "fast" physical processes in the model. At longer time horizons, the slower "feedback" processes in the Earth System begin to take a more prominent role in the accuracy of the forecast. Simultaneously, forecast-skill attribution transitions away from feature-based metrics that emphasize smaller scales (e.g., representations of fronts and vortices) to metrics that emphasize the statistical distributions of large-scale features (e.g., ENSO diagnostics and teleconnections). This presentation will summarize studies performed using the GEOS-S2S (subseasonal to seasonal) system that explore the impacts of NASA observations on the fidelity of the forecasts. The GEOS-S2S system is configured for the atmosphere-ocean-land-ice model and is initialized using in-situ and space-based observations, including atmospheric aerosols and ozone which are not typically analyzed in such systems. The GEOS-S2S model routinely includes aerosol feedbacks, which systematically impact the realism of the forecasts, providing a first example of how suitable NASA observations impact the performance of the GEOS-S2S system. Studies in which a stratospheric chemistry module is activated in the GEOS-S2S system allow the impacts of ozone radiative feedbacks to be isolated. Space-based observations of sea-surface temperature and altimetry are routinely analyzed for the initialization of the GEOS-S2S system; recent advances allow the use of NASA's sea-surface salinity data, which are shown to impact the long-range skill of the forecasts.

Pawson, Steven↗
NASA NTRS1991

Intelligent tutoring systems for systems engineering methodologies

The general goal is to provide the technology required to build systems that can provide intelligent tutoring in IDEF (Integrated Computer Aided Manufacturing Definition Method) modeling. The following subject areas are covered: intelligent tutoring systems for systems analysis methodologies; IDEF tutor architecture and components; developing cognitive skills for IDEF modeling; experimental software; and PC based prototype.

Meyer, Richard J.↗
NASA NTRS2022

Lightning Over Central Canada: Skill Assessment for Various Land-Atmosphere Model Configurations and Lightning Indices Over a Boreal Study Area

Current lightning predictions are uncertain because they rely on empirical diagnostic relationships and often use coarse-scale climate scenario simulations in which deep convection is parameterized. Previous studies demonstrated that simulations with convection permitting resolutions improve lightning predictions compared to coarser-grid simulations using convection parameterizations for different geographical locations but not over the boreal zone. In this study, lightning simulations with the NASA Unified-Weather Research and Forecasting (NU-WRF) model are evaluated over a domain including the Great Slave Lake in Canada, for six lightning seasons. The simulations are performed at convection parameterized (9 km) and convection-permitting (3 km) resolution using the Goddard 4ICE and the Thompson microphysics schemes. Four lightning indices are evaluated against observations from the Canadian Lightning Detection Network, in terms of spatiotemporal frequency distribution, spatial pattern, daily climatology, and an event-based overall skill assessment. The Thompson scheme is, regardless of the spatial resolution, superior to the Goddard 4ICE scheme in predicting daily climatology but worse in predicting the spatial patterns of lightning occurrence. Results indicate that lightning estimation benefits from modeling at convection-permitting resolution, in particular for the ice based lightning indices. In contrast, the product of convective available potential energy and precipitation rate proved to be the most robust index that was largely invariant to varying spatial resolution. Finally, this study reveals issues of the models to reproduce the observed spatial pattern of lightning well, which might be related to an insufficient representation of land surface heterogeneity, including peatlands, in the study area.

Jonas Mortelmans↗
NASA NTRS2022

Advancing Solar Energetic Particle Forecasting

With growing interest from the aviation and satellite industries, and for NASA’s upcoming Artemis lunar missions, the need for improved scientific understanding and accurate forecasting of solar energetic particle events has never been stronger. In this paper we discuss the observational, validation and model transition support required to achieve these goals. Well-calibrated, high-quality energetic electron, proton, and ion measurements are essential. Expansions to the fields of view offered by current X-ray, extreme ultraviolet and coronagraph instruments, to obtain increased coverage of the solar corona and heliosphere, from vantage points off the Sun-Earth line, are desired for model input. New observations of suprathermal particles are needed to characterize seed particle distributions and low latency space-based observations of solar radio emissions are also desired. Together, this observational suite should offer high cadence, low latency, reliable and accurate space weather data streams. Consistent, extensive and quantitative model validation is required to assess scientific advancements and pave the way for models transitioning to real-time forecast operations. Model performance and skill should be compared to observations and to current operational forecasting baselines. Finally, resources are required to support the significant effort of transitioning mature models into forecast operations.

solar energetic particles↗
NASA NTRS1992

A survey of commercial object-oriented database management systems

The object-oriented data model is the culmination of over thirty years of database research. Initially, database research focused on the need to provide information in a consistent and efficient manner to the business community. Early data models such as the hierarchical model and the network model met the goal of consistent and efficient access to data and were substantial improvements over simple file mechanisms for storing and accessing data. However, these models required highly skilled programmers to provide access to the data. Consequently, in the early 70's E.F. Codd, an IBM research computer scientists, proposed a new data model based on the simple mathematical notion of the relation. This model is known as the Relational Model. In the relational model, data is represented in flat tables (or relations) which have no physical or internal links between them. The simplicity of this model fostered the development of powerful but relatively simple query languages that now made data directly accessible to the general database user. Except for large, multi-user database systems, a database professional was in general no longer necessary. Database professionals found that traditional data in the form of character data, dates, and numeric data were easily represented and managed via the relational model. Commercial relational database management systems proliferated and performance of relational databases improved dramatically. However, there was a growing community of potential database users whose needs were not met by the relational model. These users needed to store data with data types not available in the relational model and who required a far richer modelling environment than that provided by the relational model. Indeed, the complexity of the objects to be represented in the model mandated a new approach to database technology. The Object-Oriented Model was the result.

Atkins, John↗
NASA NTRS1981

Integrated and spectral energetics of the GLAS general circulation model

Integrated and spectral error energetics of the Goddard Laboratory for Atmospheric Sciences (GLAS) general circulation model are compared with observations for periods in January 1975, 1976, and 1977. For two cases the model shows significant skill in predicting integrated energetics quantities out to two weeks, and for all three cases, the integrated monthly mean energetics show qualitative improvements over previous versions of the model in eddy kinetic energy and barotropic conversions. Fundamental difficulties remain with leakage of energy to the stratospheric level. General circulation model spectral energetics predictions are compared with the corresponding observational spectra on a day by day basis. Eddy kinetic energy can be correct while significant errors occur in the kinetic energy of wavenumber three. Single wavenumber dominance in eddy kinetic energy and the correlation of spectral kinetic and potential energy are demonstrated.

Tenenbaum, J.↗
NASA NTRS1982

Integrated and spectral energetics of the GLAS general circulation model

Integrated and spectral error energetics of the GLAS General circulation model are compared with observations for periods in January 1975, 1976, and 1977. For two cases the model shows significant skill in predicting integrated energetics quantities out to two weeks, and for all three cases, the integrated monthly mean energetics show qualitative improvements over previous versions of the model in eddy kinetic energy and barotropic conversions. Fundamental difficulties remain with leakage of energy to the stratospheric level, particularly above strong initial jet streams associated in part with regions of steep terrain. The spectral error growth study represents the first comparison of general circulation model spectral energetics predictions with the corresponding observational spectra on a day by day basis. The major conclusion is that eddy kinetics energy can be correct while significant errors occur in the kinetic energy of wavenumber 3. Both the model and observations show evidence of single wavenumber dominance in eddy kinetic energy and the correlation of spectral kinetics and potential energy.

Tenenbaum, J.↗
NASA NTRS2018

The SMAP Level-4 ECO Product - Phase 1: Improving Vegetation Simulations Through Observation-Driven Parameter Estimation

Simulations of hydrological fields as well as water, energy and carbon fluxes from the land surface to the atmosphere are crucial for a wide range of applications, including agricultural advisories, forecasts of (short-term) atmospheric behavior and seasonal weather predictions including forecasts of extreme events, such as heatwaves or droughts. The NASA Soil Moisture Active Passive (SMAP) mission Level-4 (L4) Eco-Hydrology (ECO) project aims to improve modeled estimates of the terrestrial water, energy and carbon fluxes and states by developing a fully-coupled hydrology-vegetation data assimilation (DA) algorithm. The DA system is developed for the NASA Goddard Earth Observing System version 5 (GEOS-5) Catchment-CN land surface model, which combines land hydrology components of the GEOS-5 Catchment model with dynamic vegetation components of the Community Land Model version 4. Catchment-CN fully couples the terrestrial water, energy and carbon cycles, allowing feedbacks from the land hydrology to the biosphere and vice versa. For SMAP L4 ECO a calibration of the Catchment-CN vegetation parameterization against observations of the fraction of absorbed photosynthetically active radiation (FPAR) from the Moderate Resolution Imaging Spectroradiometer (MODIS) is implemented to improve the model's standalone skill. Next, the DA algorithm used to produce the SMAP L4 soil moisture product is adapted to Catchment-CN to assimilate SMAP brightness temperatures and inform the model's land hydrology component. The DA system is further extended to assimilate MODIS FPAR observations in order to constrain the model's dynamic vegetation component. In this presentation, we demonstrate that the Catchment-CN parameter calibration leads to more realistic vegetation simulations and reduces the root mean squared error between modeled and observed vegetation states across the model's various plant functional types. We also show that the assimilation of SMAP observations is able to improve the average correlation, bias and unbiased RMSE between the modeled surface and root zone soil moisture estimates, and ground observations from the SMAP core validation sites.

Kolassa, J.↗
NASA NTRS1993

The effects of a dynamic graphical model during simulation-based training of console operation skill

LOADER is a Windows-based simulation of a complex procedural task. The task requires subjects to execute long sequences of console-operation actions (e.g., button presses, switch actuations, dial rotations) to accomplish specific goals. The LOADER interface is a graphical computer-simulated console which controls railroad cars, tracks, and cranes in a fictitious railroad yard. We hypothesized that acquisition of LOADER performance skill would be supported by the representation of a dynamic graphical model linking console actions to goal and goal states in the 'railroad yard'. Twenty-nine subjects were randomly assigned to one of two treatments (i.e., dynamic model or no model). During training, both groups received identical text-based instruction in an instructional-window above the LOADER interface. One group, however, additionally saw a dynamic version of the bird's-eye view of the railroad yard. After training, both groups were tested under identical conditions. They were asked to perform the complete procedure without guidance and without access to either type of railroad yard representation. Results indicate that rather than becoming dependent on the animated rail yard model, subjects in the dynamic model condition apparently internalized the model, as evidenced by their performance after the model was removed.

Farquhar, John D.↗
NASA NTRS2018

The SMAP Level-4 ECO Project: Improving Terrestrial Flux Estimates Through Coupled Hydrology-Vegetation Data Assimilation

Simulations of hydrologic and vegetation states as well as water, energy and carbon fluxes from the land surface to the atmosphere are crucial for a wide range of applications, including agricultural advisories, forecasts of (short-term) atmospheric behavior and seasonal weather predictions including forecasts of extreme events, such as heatwaves or droughts. The NASA Soil Moisture Active Passive (SMAP) mission Level-4 Eco-Hydrology (L4-ECO) project aims to improve modeled estimates of the terrestrial water, energy and carbon fluxes and states by developing a fully-coupled hydrology-vegetation data assimilation system. This system is developed around the NASA Goddard Earth Observing System (GEOS) Catchment-CN land surface model, which combines land hydrology and energy balance components of the GEOS Catchment model with dynamic vegetation components of the Community Land Model version 4. Catchment-CN fully couples the terrestrial water, energy and carbon cycles, allowing feedbacks from the land hydrology to the biosphere and vice versa.Here, we implement a calibration of the Catchment-CN vegetation parameterization against observations of the fraction of absorbed photosynthetically active radiation (FPAR) from the Moderate Resolution Imaging Spectroradiometer (MODIS) to improve the model's standalone skill. Later, the DA algorithm used to produce the SMAP L4 soil moisture product will be adapted to Catchment-CN to assimilate SMAP brightness temperatures and inform the model's land hydrology component. Finally, the DA system will be further extended to assimilate MODIS FPAR observations in order to constrain the model's dynamic vegetation component.In this presentation, we demonstrate that the Catchment-CN parameter calibration leads to more realistic vegetation simulations and reduces the root mean squared error between modeled and observed vegetation states across the model's various plant functional types. We also show that the assimilation of SMAP observations is able to improve the average correlation, bias and unbiased RMSE between the modeled surface and root zone soil moisture estimates, and ground observations from the SMAP core validation sites.

Kolassa, Jana↗
NASA NTRS2004

A Case Study of the Impact of AIRS Temperature Retrievals on Numerical Weather Prediction

Large errors in numerical weather prediction are often associated with explosive cyclogenesis. Most studes focus on the under-forecasting error, i.e. cases of rapidly developing cyclones which are poorly predicted in numerical models. However, the over-forecasting error (i.e., to predict an explosively developing cyclone which does not occur in reality) is a very common error that severely impacts the forecasting skill of all models and may also present economic costs if associated with operational forecasting. Unnecessary precautions taken by marine activities can result in severe economic loss. Moreover, frequent occurrence of over-forecasting can undermine the reliance on operational weather forecasting. Therefore, it is important to understand and reduce the prdctions of extreme weather associated with explosive cyclones which do not actually develop. In this study we choose a very prominent case of over-forecasting error in the northwestern Pacific. A 960 hPa cyclone develops in less than 24 hour in the 5-day forecast, with a deepening rate of about 30 hPa in one day. The cyclone is not versed in the analyses and is thus a case of severe over-forecasting. By assimilating AIRS data, the error is largely eliminated. By following the propagation of the anomaly that generates the spurious cyclone, it is found that a small mid-tropospheric geopotential height negative anomaly over the northern part of the Indian subcontinent in the initial conditions, propagates westward, is amplified by orography, and generates a very intense jet streak in the subtropical jet stream, with consequent explosive cyclogenesis over the Pacific. The AIRS assimilation eliminates this anomaly that may have been caused by erroneous upper-air data, and represents the jet stream more correctly. The energy associated with the jet is distributed over a much broader area and as a consequence a multiple, but much more moderate cyclogenesis is observed.

Reale, O.↗
NASA NTRS2013

The Role of Model and Initial Condition Error in Numerical Weather Forecasting Investigated with an Observing System Simulation Experiment

A series of experiments that explore the roles of model and initial condition error in numerical weather prediction are performed using an observing system simulation experiment (OSSE) framework developed at the National Aeronautics and Space Administration Global Modeling and Assimilation Office (NASA/GMAO). The use of an OSSE allows the analysis and forecast errors to be explicitly calculated, and different hypothetical observing networks can be tested with ease. In these experiments, both a full global OSSE framework and an 'identical twin' OSSE setup are utilized to compare the behavior of the data assimilation system and evolution of forecast skill with and without model error. The initial condition error is manipulated by varying the distribution and quality of the observing network and the magnitude of observation errors. The results show that model error has a strong impact on both the quality of the analysis field and the evolution of forecast skill, including both systematic and unsystematic model error components. With a realistic observing network, the analysis state retains a significant quantity of error due to systematic model error. If errors of the analysis state are minimized, model error acts to rapidly degrade forecast skill during the first 24-48 hours of forward integration. In the presence of model error, the impact of observation errors on forecast skill is small, but in the absence of model error, observation errors cause a substantial degradation of the skill of medium range forecasts.

Model Error↗
NASA NTRS2014

A Practical Philosophy of Complex Climate Modelling

We give an overview of the practice of developing and using complex climate models, as seen from experiences in a major climate modelling center and through participation in the Coupled Model Intercomparison Project (CMIP).We discuss the construction and calibration of models; their evaluation, especially through use of out-of-sample tests; and their exploitation in multi-model ensembles to identify biases and make predictions. We stress that adequacy or utility of climate models is best assessed via their skill against more naive predictions. The framework we use for making inferences about reality using simulations is naturally Bayesian (in an informal sense), and has many points of contact with more familiar examples of scientific epistemology. While the use of complex simulations in science is a development that changes much in how science is done in practice, we argue that the concepts being applied fit very much into traditional practices of the scientific method, albeit those more often associated with laboratory work.

complex simulation↗
NASA NTRS2015

Use of North American and European Air Quality Networks to Evaluate Global Chemistry-Climate Modeling of Surface Ozone

We test the current generation of global chemistry-climate models in their ability to simulate observed, present-day surface ozone. Models are evaluated against hourly surface ozone from 4217 stations in North America and Europe that are averaged over 1 degree by 1 degree grid cells, allowing commensurate model-measurement comparison. Models are generally biased high during all hours of the day and in all regions. Most models simulate the shape of regional summertime diurnal and annual cycles well, correctly matching the timing of hourly (approximately 15:00 local time (LT)) and monthly (mid-June) peak surface ozone abundance. The amplitude of these cycles is less successfully matched. The observed summertime diurnal range (25 ppb) is underestimated in all regions by about 7 parts per billion, and the observed seasonal range (approximately 21 parts per billion) is underestimated by about 5 parts per billion except in the most polluted regions, where it is overestimated by about 5 parts per billion. The models generally match the pattern of the observed summertime ozone enhancement, but they overestimate its magnitude in most regions. Most models capture the observed distribution of extreme episode sizes, correctly showing that about 80 percent of individual extreme events occur in large-scale, multi-day episodes of more than 100 grid cells. The models also match the observed linear relationship between episode size and a measure of episode intensity, which shows increases in ozone abundance by up to 6 parts per billion for larger-sized episodes. We conclude that the skill of the models evaluated here provides confidence in their projections of future surface ozone.

diurnal variations↗