Search NASA⌕ Search

SEARCH · Search NASA

Results for “Prediction skill”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

A commentary on perception-action relationships in spatial display instruments

Transfer of information across disciplines is promoted, while basic and applied researchers are cautioned about the danger of assuming simple relationships between stimulus information, perceptual impressions, and performance including pattern recognition and sensorimotor skills. A theoretical and empirical foundation was developed predicting those relationships.

Shebilske, Wayne L.↗

Short–Term Forecasting of Wind Gusts at Airports Across CONUS Using Machine Learning

Short–term forecasting of wind gusts, particularly those of higher intensity, is of great societal importance but is challenging due to the presence of multiple gust generation mechanisms. Wind gust observations from eight high–passenger–volume airports across the continental United States (CONUS) are summarized and used to develop predictive models of wind gust occurrence and magnitude. These short–term (same hour) forecast models are built using multiple logistic and linear regression, as well as artificial neural networks (ANNs) of varying complexity. A suite of 19 upper–air predictors drawn from the ERA5 reanalysis and an autoregressive (AR) term are used. Stepwise procedures instruct predictor selection, and resampling is used to quantify model stability. All models are developed separately for the warm (April–September) and cold (October–March) seasons. Results show that ANNs of 3–5 hidden layers (HLs) generally exhibit higher hit rates than logistic regression models and also improve skill with respect to wind gust magnitudes. However, deeper networks with more HLs increase false alarm rates in occurrence models and mean absolute error in magnitude models due to model overfitting. For model skill, inclusion of the AR term is critical while the majority of the remaining skill derives from wind speeds and lapse rates. A predictive ceiling is also clearly demonstrated, particularly for the strong and damaging gust magnitudes, which appears to be partially due to ERA5 predictor characteristics and the presence of mixed wind climates.

54 ENVIRONMENTAL SCIENCES↗

Multi-Variate LSTM Prediction of Alaska Magnetometer Chain Utilizing a Coupled Model Approach

During periods of rapidly changing geomagnetic conditions electric fields form within the Earth’s surface and induce currents known as geomagnetically induced currents(GICs), which interact with unprotected electrical systems our society relies on. In this study, we train multi-variate Long-Short Term Memory neural networks to predict magnitude of north-south component of the geomagnetic field (|BN|) at multiple ground magnetometer stations across Alaska provided by the SuperMAG database with a future goal of predicting geomagnetic field disturbances. Each neural network is driven by solar wind and interplanetary magnetic field inputs from the NASA OMNI database spanning from 2000–2015 and is fine tuned for each station to maximize the effectiveness in predicting |BN|. The neural networks are then compared against multivariate linear regression models driven with the same inputs at each station using Heidke skill scores with thresholds at the 50, 75, 85, and 99 percentiles for |BN|. The neural network models show significant increases over the linear regression models for |BN| thresholds. We also calculate the Heidke skill scores for d|BN|/dt by deriving d|BN|/dt from |BN| predictions. However, neural network models do not show clear outperformance compared to the linear regression models. To retain the sign information and thus predict BN instead of |BN|, a secondary so-called polarity model is utilized. The polarity model is run in tandem with the neural networks predicting geomagnetic field in a coupled model approach and results in a high correlation between predicted and observed values for all stations. We find this model a promising starting point for a machine learned geomagnetic field model to be expanded upon through increased output time history and fast turnaround times.

Matthew Blandin↗

Optimizing Seasonal‐To‐Decadal Analog Forecasts With a Learned Spatially‐Weighted Mask

Abstract Seasonal‐to‐decadal climate prediction is crucial for decision‐making in a number of industries, but forecasts on these timescales have limited skill. Here, we develop a data‐driven method for selecting optimal analogs for seasonal‐to‐decadal analog forecasting. Using an interpretable neural network, we learn a spatially‐weighted mask that quantifies how important each grid point is for determining whether two climate states will evolve similarly. We show that analogs selected using this weighted mask provide more skillful forecasts than analogs that are selected using traditional spatially‐uniform methods. This method is tested on two prediction problems using the Max Planck Institute for Meteorology Grand Ensemble: multi‐year prediction of North Atlantic sea surface temperatures, and seasonal prediction of El Niño Southern Oscillation. This work demonstrates a methodical approach to selecting analogs that may be useful for improving seasonal‐to‐decadal forecasts and understanding their sources of skill.

Geology↗

Investigation of Coupled Land-Atmosphere Carbon Dynamics and Seasonal Carbon Forecast Skill

In this talk, I will present recent efforts to explore coupled land-atmosphere carbon dynamics and carbon cycle predictability. First, the impact of a regional drought on land and atmospheric carbon was studied by imposing an idealized spring drought in coupled land‐atmosphere ensemble simulations. Through drought-induced impact on remote meteorology, the drought alters land’s productivity not only in the drought area but also in the adjacent areas. The atmospheric CO2 anomalies extend to an area up to three times of that of the imposed drought, which suggests that atmospheric transport needs to be considered in the interpretation of drought‐induced carbon anomalies. Increase in column‐averaged monthly CO2 by the imposed drought is at the edge of the uncertainty from single soundings of current greenhouse gas observing satellites. Secondly, the seasonal carbon forecast skill was explored using NASA’s subseasonal-to-seasonal (S2S) ensemble forecast and a terrestrial biosphere model. The result demonstrates an ability to accurately predict spring-summer carbon uptake at multi-month leads in the Northern Hemisphere mid- and high latitude land. The skill appears to be achieved by accurate forecasts of snow removal timing as well as proper initialization of carbon and nitrogen states. Additionally, I will briefly discuss other ongoing research activities to improve hydrometeorological prediction by integrating biogeochemical processes, to evaluate forecast skill of wildfire, and to apply subseasonal forecasts to water resources management.

Eunjee Lee↗

The GLACE-2 Experiment

A major motivation for the study of the coupled land-atmosphere system is the idea that soil moisture anomalies may affect future meteorological variables through their effects on future surface energy and water budgets. If true, the accurate initialization of soil moisture in a subseasonal or seasonal forecast system may improve forecast skill, making the forecast products more valuable to society. The Global Land-Atmosphere Coupling Experiment (GLACE-2) project is examining, with a wide variety of models, the degree to which subseasonal (out to two months) precipitation and air temperature forecasts improve through the realistic initialization of soil moisture. For the first time ever, a global consensus should emerge regarding the value of land initialization for forecasts, perhaps motivating national forecast centers to make full use of land moisture initialization in their operations. Participants in GLACE-2 perform two series of forecasts, each consisting of 100 2-month forecast ensembles (10 members per ensemble) covering ten boreal spring and summer start-dates in each of the years 1986-1995. Series 1 utilizes realistic land surface state initialization, provided through a decadal offline simulation using realistic meteorological forcing, as provided by the Global Soil Wetness Project - Phase 2 (GSWP-2), a research activity of the Global Land-Atmosphere System Study (GLASS) of GEWEX. Series 2 is identical to Series 1 in every way except for the fact that it does not benefit from realistic land state initialization. Through the comparison of Series 1 and 2, we isolate the impact of land initialization on the forecasts. Optional extensions to these base runs include forecasts covering additional years, using alternative meteorological forcing for the land initialization. To date, GLACE-2 has garnered participation from eleven modeling groups, covering 13 atmospheric models. Analysis of available results is already well underway, both at the individual modeling institutions and at the NASA Goddard Space Flight Center, which is coordinating the project. Analysis focuses on two elements of the forecast problem: (i) the quantification of model-specific "predictability" (i.e., the degree to which simulated atmospheric chaos will foil a forecast, even under the assumption of"perfect" model physics, initialization data, and validation data) and its decay with time; and (ii) the quantification of forecast skill, determined through a comparison of predicted precipitation and air temperature against observations. Indeed, the specific contribution of land initialization to both these elements is isolated through a comparison of the Series 1 and 2 forecasts. We examine the two elements at four different forecast leads: 1-15, 16-30, 31-45, and 46-60 days. Statistics-based approaches for enhancing skill (essentially using observational statistics to reduce the impact of the models' climatic biases) are also tested. In the present talk, we provide an update of progress in GLACE-2, featuring quantifications of predictability and forecast skill for a number of the participating models and providing a "first look" at the desired consensus view of land impacts on subseasonal forecasts.

Koster, Randal↗

Upgrade Summer Severe Weather Tool

The goal of this task was to upgrade to the existing severe weather database by adding observations from the 2010 warm season, update the verification dataset with results from the 2010 warm season, use statistical logistic regression analysis on the database and develop a new forecast tool. The AMU analyzed 7 stability parameters that showed the possibility of providing guidance in forecasting severe weather, calculated verification statistics for the Total Threat Score (TTS), and calculated warm season verification statistics for the 2010 season. The AMU also performed statistical logistic regression analysis on the 22-year severe weather database. The results indicated that the logistic regression equation did not show an increase in skill over the previously developed TTS. The equation showed less accuracy than TTS at predicting severe weather, little ability to distinguish between severe and non-severe weather days, and worse standard categorical accuracy measures and skill scores over TTS.

Watson, Leela↗

Large‐Scale Statistically Meaningful Patterns (LSMPs) Associated With Precipitation Extremes Over Northern California

Abstract We analyze large‐scale statistically meaningful patterns (LSMPs) that precede extreme precipitation (PEx) events over Northern California (NorCal). We find LSMPs by applying k‐means clustering to the two leading principal components of daily 500 hPa geopotential height anomalies two days before the onset, from October to March during 1948–2015. Statistical significance testing based on Monte Carlo simulations suggests a minimum of four statistically distinguished LSMP clusters. The four LSMP clusters are characterized as Northwest continental negative height anomaly, Eastward positive “Pacific‐North American Pattern (PNA),” Westward negative “PNA,” and Prominent Alaskan ridge. These four clusters, shown in multiple variables, evolve very differently and have differing links to the Arctic and tropical Pacific regions. Using binary forecast skill measures and a new copula‐based framework for predicting PEx events, we find LSMP indices that are useful predictors of NorCal PEx events, with moisture‐based variables being the best predictors of PEx events at least 6 days before the onset, and the lower atmospheric variables being better than their upper atmospheric counterparts any day in advance tested. To ensure statistical rigor, the LSMPs analyzed here (with the modified acronym) include local tests of both significance and consistency, which are not always featured in the literature on large‐scale meteorological patterns.

54 ENVIRONMENTAL SCIENCES↗

Bioaerosols are the dominant source of warm-temperature immersion-mode INPs and drive uncertainties in INP predictability

Ice-nucleating particles (INPs) are rare atmospheric aerosols that initiate primary ice formation, but accurately simulating their concentrations and variability in large-scale climate models remains a challenge. Doing so requires both simulating major particle sources and parameterizing their ice nucleation (IN) efficiency. Validating and improving model predictions of INP concentrations requires measuring their concentrations delineated by particle type. We present a method to speciate INP concentrations into contributions from dust, sea spray aerosol (SSA), and bioaerosol. Field campaign data from Bodega Bay, California, showed that bioaerosols were the primary source of INPs between –12° and –20°C, while dust was a minor source and SSA had little impact. We found that recent parameterizations for dust and SSA accurately predicted ambient INP concentrations. However, the model did not skillfully simulate bioaerosol INPs, suggesting a need for further research to identify major factors controlling their emissions and INP efficiency for improved representation in models.

54 ENVIRONMENTAL SCIENCES↗

Assimilation of AERONET and MODIS AOT Observations Using Variational and Ensemble Data Assimilation Methods and Its Impact on Aerosol Forecasting Skill

Data assimilation of Aerosol Robotic Network (AERONET) and Moderate Resolution Imaging Spectroradiometer (MODIS) aerosol optical thickness (AOT) for aerosol forecasting was tested within the Navy Aerosol Analysis Prediction System (NAAPS) framework, using variational and ensemble data assimilation methods. Navy aerosol forecasting currently makes use of a deterministic NAAPS simulation coupled to Navy Variational Data Assimilation System for aerosol optical depth, a two-dimensional variational data assimilation system, for MODIS AOT assimilation. An ensemble version of NAAPS (ENAAPS) coupled to an ensemble adjustment Kalman filter (EAKF) from the Data Assimilation Research Testbed was recently developed, allowing for a range of data assimilation and forecasting experiments to be run with deterministic NAAPS and ENAAPS. The main findings are that the EAKF, with its flow-dependent error covariances, makes better use of sparse observations such as AERONET AOT. Assimilating individual AERONET observations in the two-dimensional variational system can increase the analysis errors when observations are located in high AOT gradient regions. By including AERONET with MODIS AOT assimilation, the magnitudes of peak aerosol events (AOT> 1) were better captured with improved temporal variability, especially in India and Asia where aerosol prediction is a challenge. Assimilating AERONET AOT with MODIS had little impact on the 24 h forecast skill compared to MODIS assimilation only, but differences were found downwind of AERONET sites. The 24 h forecast skill was approximately the same for forecasts initialized with analyses from AERONET AOT assimilation alone compared to MODIS assimilation, particularly in regions where the AERONET network is dense; including the United States and Europe, indicating that AERONET could serve as a backup observation network for over-land synoptic-scale aerosol events.

Rubin, Juli I.↗

Skill of Mesoscale Models in Forecasting Springtime Macrophysical Cloud Properties at the Savannah River Site in the Southeastern US

Predicting boundary layer clouds is important for the accurate modeling of pollutant dispersion. Higher resolution mesoscale models would be expected to produce better forecasts of cloud properties that affect dispersion. Using ceilometer observations, we assess the skill of two operational mesoscale models (RAMS and WRF) to forecast cloud base altitude and cloud fraction at the Savannah River Site in the southeastern US during the springtime. Verifications were performed at small spatial and temporal scales necessary for dispersion modeling. Both models were unreliable with a 50% (RAMS) and a 46% (WRF) rate of predicting clouds observed by the ceilometer which led to low cloud fraction predictions. Results indicated that WRF better predicted daytime cloud bases from convection that occurred frequently later in the period and RAMS better predicted nighttime cloud bases. Using root mean squared error (RMSE) to score the forecast periods also highlighted this diurnal dichotomy, with WRF scores better during the day and RAMS scores better at night. Analysis of forecast errors revealed divergent model cloud base biases—WRF low and RAMS high. A hybrid solution which weighs more heavily the RAMS nighttime forecasts and WRF daytime forecasts will likely provide the best prediction of cloud properties for dispersion.

54 ENVIRONMENTAL SCIENCES↗

Distortion Representation of Forecast Errors for Model Skill Assessment and Objective Analysis

We proposed a novel characterization of errors for numerical weather predictions. In its simplest form we decompose the error into a part attributable to phase errors and a remainder. The phase error is represented in the same fashion as a velocity field and is required to vary slowly and smoothly with position. A general distortion representation allows for the displacement and amplification or bias correction of forecast anomalies. Characterizing and decomposing forecast error in this way has two important applications, which we term the assessment application and the objective analysis application. For the assessment application, our approach results in new objective measures of forecast skill which are more in line with subjective measures of forecast skill and which are useful in validating models and diagnosing their shortcomings. With regard to the objective analysis application, meteorological analysis schemes balance forecast error and observational error to obtain an optimal analysis. Presently, representations of the error covariance matrix used to measure the forecast error are severely limited. For the objective analysis application our approach will improve analyses by providing a more realistic measure of the forecast error. We expect, a priori, that our approach should greatly improve the utility of remotely sensed data which have relatively high horizontal resolution, but which are indirectly related to the conventional atmospheric variables. In this project, we are initially focusing on the assessment application, restricted to a realistic but univariate 2-dimensional situation. Specifically, we study the forecast errors of the sea level pressure (SLP) and 500 hPa geopotential height fields for forecasts of the short and medium range. Since the forecasts are generated by the GEOS (Goddard Earth Observing System) data assimilation system with and without ERS 1 scatterometer data, these preliminary studies serve several purposes. They (1) provide a testbed for the use of the distortion representation of forecast errors, (2) act as one means of validating the GEOS data assimilation system and (3) help to describe the impact of the ERS 1 scatterometer data.

Hoffman, Ross N.↗

Enhancing Short-Range Weather Forecasts through Temporal Variation Encoding: A Multiperiod Embedding Approach

Machine learning (ML) techniques have emerged as promising approaches to improve regional weather forecast accuracy and reliability through data-driven methods. We propose a novel ML-based weather forecasting model, the Multiperiod Embed Net (MPENet). A key distinguishing feature of MPENet is its explicit utilization of the inherent cyclic nature in weather dynamics, unlike the autoregressive strategies commonly used in other ML weather forecasting approaches. Critical cyclic structures are identified via Fourier analyses of dynamic time series. Cyclicity in the convolutional representation is achieved by transforming one-dimensional time series of meteorological variables into two-dimensional tensors based on identified periods. This approach enables the model to leverage intrinsic weather patterns, enhancing regional forecast performance. To demonstrate the effectiveness of MPENet, we conduct a comparative analysis with Nvidia’s FourCastNet. Both models are trained on High-Resolution Rapid Refresh (HRRR) data from 2015 to 2022, over a 192 km × 192 km region in Tennessee. The comparisons are performed locally at two specific locations known to have different weather dynamics due to orographic effects: Crossville, on the relatively flat Cumberland Plateau with fewer topographic airflow disruptions, and Oak Ridge, in the ridge-and-valley region, where airflow is heavily influenced by surrounding valleys and mountains. Our results indicate that FourCastNet achieves strong accuracy at very short lead times, while MPENet maintains competitive skill and shows advantages in capturing temporal evolution over longer periods. Cross-correlation analyses of MPENet and FourCastNet predictions with the HRRR data suggest that encoding critical cyclicity into the network architecture leads to improvements in the forecasting skill.

Artificial intelligence↗

Predicting Operator Execution Times Using CogTool

Researchers and developers of NextGen systems can use predictive human performance modeling tools as an initial approach to obtain skilled user performance times analytically, before system testing with users. This paper describes the CogTool models for a two pilot crew executing two different types of a datalink clearance acceptance tasks, and on two different simulation platforms. The CogTool time estimates for accepting and executing Required Time of Arrival and Interval Management clearances were compared to empirical data observed in video tapes and registered in simulation files. Results indicate no statistically significant difference between empirical data and the CogTool predictions. A population comparison test found no significant differences between the CogTool estimates and the empirical execution times for any of the four test conditions. We discuss modeling caveats and considerations for applying CogTool to crew performance modeling in advanced cockpit environments.

Santiago-Espada, Yamira↗

Managerial leadership assessment - Personality correlates of and sex differences in ratings by leaders, peers, and followers

A performance appraisal was conducted at a Fortune 500 airline. Evaluations of each manager were taken from his or her management, peers and subordinates. These ratings were related to personality clusters revealing patterns for males similar to those found between personality and performance in pilot populations. A case is made that piloting aircraft requires similar skills to managing other complex enterprises and that similar profiles predict success in each.

Gibson, Robert H.↗

Results from CrIS-ATMS Obtained Using the AIRS Science Team Retrieval Methodology

AIRS was launched on EOS Aqua in May 2002, together with AMSU-A and HSB (which subsequently failed early in the mission), to form a next generation polar orbiting infrared and microwave atmospheric sounding system. AIRS/AMSU had two primary objectives. The first objective was to provide real-time data products available for use by the operational Numerical Weather Prediction Centers in a data assimilation mode to improve the skill of their subsequent forecasts. The second objective was to provide accurate unbiased sounding products with good spatial coverage that are used to generate stable multi-year climate data sets to study the earth's interannual variability, climate processes, and possibly long-term trends. AIRS/AMSU data for all time periods are now being processed using the state of the art AIRS Science Team Version-6 retrieval methodology. The Suomi-NPP mission was launched in October 2011 as part of a sequence of Low Earth Orbiting satellite missions under the "Joint Polar Satellite System" (JPSS). NPP carries CrIS and ATMS, which are advanced infra-red and microwave atmospheric sounders that were designed as follow-ons to the AIRS and AMSU instruments. The main objective of this work is to assess whether CrIS/ATMS will be an adequate replacement for AIRS/AMSU from the perspective of the generation of accurate and consistent long term climate data records, or if improved instruments should be developed for future flight. It is critical for CrIS/ATMS to be processed using an algorithm similar to, or at least comparable to, AIRS Version-6 before such an assessment can be made. We have been conducting research to optimize products derived from CrIS/ATMS observations using a scientific approach analogous to the AIRS Version-6 retrieval algorithm. Our latest research uses Version-5.70 of the CrIS/ATMS retrieval algorithm, which is otherwise analogous to AIRS Version-6, but does not yet contain the benefit of use of a Neural-Net first guess start-up system which significantly improved results of AIRS Version-6. Version-5.70 CrIS/ATMS temperature profile and surface skin temperature retrievals are of very good quality, and are better than AIRS Version-5 retrievals, but are still significantly poorer than those of AIRS Version-6. CrIS/ATMS retrievals should improve when a Neural-Net start-up system is ready for use. We also examined CrIS/ATMS retrievals generated by NOAA using their NUCAPS retrieval algorithm, which is based on earlier versions of the AIRS Science Team retrieval algorithms. We show that the NUCAPS algorithm as currently configured is not well suited for climate monitoring purposes.

CrIS↗

Lightning Over Central Canada: Skill Assessment for Various Land-Atmosphere Model Configurations and Lightning Indices Over a Boreal Study Area

Current lightning predictions are uncertain because they rely on empirical diagnostic relationships and often use coarse-scale climate scenario simulations in which deep convection is parameterized. Previous studies demonstrated that simulations with convection permitting resolutions improve lightning predictions compared to coarser-grid simulations using convection parameterizations for different geographical locations but not over the boreal zone. In this study, lightning simulations with the NASA Unified-Weather Research and Forecasting (NU-WRF) model are evaluated over a domain including the Great Slave Lake in Canada, for six lightning seasons. The simulations are performed at convection parameterized (9 km) and convection-permitting (3 km) resolution using the Goddard 4ICE and the Thompson microphysics schemes. Four lightning indices are evaluated against observations from the Canadian Lightning Detection Network, in terms of spatiotemporal frequency distribution, spatial pattern, daily climatology, and an event-based overall skill assessment. The Thompson scheme is, regardless of the spatial resolution, superior to the Goddard 4ICE scheme in predicting daily climatology but worse in predicting the spatial patterns of lightning occurrence. Results indicate that lightning estimation benefits from modeling at convection-permitting resolution, in particular for the ice based lightning indices. In contrast, the product of convective available potential energy and precipitation rate proved to be the most robust index that was largely invariant to varying spatial resolution. Finally, this study reveals issues of the models to reproduce the observed spatial pattern of lightning well, which might be related to an insufficient representation of land surface heterogeneity, including peatlands, in the study area.

Jonas Mortelmans↗

Distortion Representation of Forecast Errors for Model Skill Assessment and Objective Analysis

We study a novel characterization of errors for numerical weather predictions. In its simplest form we decompose the error into a part attributable to phase errors and a remainder. The phase error is represented in the same fashion as a velocity field and will be required to vary slowly and smoothly with position. A general distortion representation allows for the displacement and a bias correction of forecast anomalies. In brief, the distortion is determined by minimizing the objective function by varying the displacement and bias correction fields. In the present project we use a global or hemispheric domain, and spherical harmonics to represent these fields. In this project we are initially focusing on the assessment application, restricted to a realistic but univariate 2-dimensional situation. Specifically we study the forecast errors of the 500 hPa geopotential height field for forecasts of the short and medium range. The forecasts are those of the Goddard Earth Observing System data assimilation system. Results presented show that the methodology works, that a large part of the total error may be explained by a distortion limited to triangular truncation at wavenumber 10, and that the remaining residual error contains mostly small spatial scales.

Hoffman, Ross N.↗