SEARCH · Search NASA

Results for “Prediction skill”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Evaluation of Subseasonal-to-Seasonal (S2S) precipitation forecast from the North American Multi-Model ensemble phase II (NMME-2) over the contiguous U.S.

The second phase of the North America Multi-Model Ensemble (NMME-2) provides globally available Subseasonal-to-Seasonal (S2S) precipitation forecasts with a daily resolution. The S2S precipitation forecasts are getting increasing attention for their potentials in providing hydrometeorological forcing information for water resources planning at an extended range. However, the forecast skills of many existing S2S forecast products will significantly decrease when the lead time increases, hindering their applicability for watershed-scale hydrologic modeling. Therefore, forecast validation and large-scale evaluation are of great importance for water resources planning and hydrological applications. In this study, we comprehensively evaluate the S2S precipitation forecasts from the NMME-2 dataset over the contiguous United States (CONUS) and during the study period from 1982 to 2011. Three aspects of precipitation forecast capabilities are compared and analyzed: bias, skill scores, and the ability to predict extreme precipitation events. The Parameter-elevation Regressions on Independent Slopes Model (PRISM) is used as ground truth reference. Differs from other regional forecast validation study, we further examined and analyzed the dependences of NMME-2 precipitation forecast skills according to different seasonality, geographical locations, and lead times. Results show that the forecast biases are not sensitive to lead times but are seasonally dependent of all NMME-2 models. Overestimations are found in the Western U.S. in cooler seasons while underestimations are observed in the central regions of the U.S. in warmer seasons. The forecast skill of all individual NMME-2 models generally decreases as increases of lead times. The simple model averaging (SMA) of five NMME-2 models demonstrates a higher forecast skill than any individual NMME-2 models. Spatially, the highest forecast skill scores are observed at coastal areas in the Western U.S. with an one-week lead time. As compared to the historical resampled forecasts, NMME-2 also shows better performance in predicting extreme precipitation events above 99% percentiles and below 1% percentiles with higher probability of detections and lower false alarm ratios. Finally, the obtained results suggest the great potentials of NMME-2 precipitation forecasts in assisting ensemble hydrologic forecasts at the S2S scale over the CONUS.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2001

Dynamical Interpolation of Mesoscale Flows in the TOPEX/Poseidon Diamond Surrounding the U.S. Joint Global Ocean Flux Study Bermuda Atlantic Time-Series Study Site

An open boundary ocean model is configured in a domain bounded by the four TOPEX/Poseidon (T/P) ground tracks surrounding the US Joint Global Ocean Flux Study Bermuda Atlantic Time-Series Study (BATS) site. This implementation facilitates prescription of model boundary conditions directly from altimetric measurements (both TIP and ERS-2). The expected error characteristics for a domain of this size with periodically updated boundary conditions are established with idealized numerical experiments using simulated data. A hindcast simulation is then constructed using actual altimetric observations during the period October 1992 through September 1998. Quantitative evaluation of the simulation suggests significant skill. The correlation coefficient between predicted sea level anomaly and ERS observations in the model interior is 0.89; that for predicted versus observed dynamic height anomaly based on hydrography at the BATS site is 0.73. Comparison with the idealized experiments suggests that the main source of error in the hindcast is temporal undersampling of the boundary conditions. The hindcast simulation described herein provides a basis for retrospective analysis of BATS observations in the context of the mesoscale eddy field.

McGillicuddy, Dennis J., Jr.↗

NASA NTRS2001

Dynamical Interpolation of Mesoscale Flows in the TOPEX/ Poseidon Diamond Surrounding the U.S. Joint Global Ocean Flux Study Bermuda Atlantic Time-Series Study Site

An open boundary ocean model is configured in a domain bounded by the four TOPEX/Poseidon (TIP) ground tracks surrounding the U.S. Joint Global Ocean Flux Study Bermuda Atlantic Time-series Study (BATS) site. This implementation facilitates prescription of model boundary conditions directly from altimetric measurements (both TIP and ERS-2). The expected error characteristics for a domain of this size with periodically updated boundary conditions are established with idealized numerical experiments using simulated data. A hindcast simulation is then constructed using actual altimetric observations during the period October 1992 through September 1998. Quantitative evaluation of the simulation suggests significant skill. The correlation coefficient between predicted sea level anomaly and ERS observations in the model interior is 0.89; that for predicted versus observed dynamic height anomaly based on hydrography at the BATS site is 0.73. Comparison with the idealized experiments suggests that the main source of error in the hindcast is temporal undersampling of the boundary conditions. The hindcast simulation described herein provides a basis for retrospective analysis of BATS observations in the context of the mesoscale eddy field.

McGillicuddy, D. J.↗

NASA NTRS2010

Assessment of the Breakup of the Antarctic Polar Vortex in Two New Chemistry-Climate Models

Successful simulation of the breakup of the Antarctic polar vortex depends on the representation of tropospheric stationary waves at Southern Hemisphere middle latitudes. This paper assesses the vortex breakup in two new chemistry-climate models (CCMs). The stratospheric version of the UK Chemistry and Aerosols model is able to reproduce the observed timing of the vortex breakup. Version 2 of the Goddard Earth Observing System (GEOS V2) model is typical of CCMs in that the Antarctic polar vortex breaks up too late; at 10 hPa, the mean transition to easterlies at 60 S is delayed by 12-13 days as compared with the ERA-40 and National Centers for Environmental Prediction reanalyses. The two models' skill in simulating planetary wave driving during the October-November period accounts for differences in their simulation of the vortex breakup, with GEOS V2 unable to simulate the magnitude and tilt of geopotential height anomalies in the troposphere and thus underestimating the wave driving. In the GEOS V2 CCM the delayed breakup of the Antarctic vortex biases polar temperatures and trace gas distributions in the upper stratosphere in November and December.

Hurwitz, M. M.↗

NASA NTRS2020

Soil Moisture as a Harbinger of Increased Forecast Reliability at Subseasonal Time Scales

The shape of the nonlinear relationship between evapotranspiration and soil moisture (the "ET-W relationship") helps control the evolution of soil moisture with time. Together, the shape of the relationship and the magnitude of the soil moisture anomaly at the beginning of a subseasonal forecast help determine whether a given anomaly will still be present at subseasonal leads, allowing it to contribute to skill in subseasonal temperature and precipitation prediction at those leads. In this study we examine subseasonal prediction in the context of soil moisture initialization using a suite of forecasts performed with the NASA GEOS seasonal forecast system. Large soil moisture anomalies are in fact found to be harbingers of increased skill in the subseasonal forecasts. Furthermore, accounting explicitly for the nonlinear shape of the ET-W relationship improves our ability to quantity the increase in forecast reliability associated with soil moisture initialization.

Koster, Randal↗

DOE OSTI2023

Using Ultrasound Image Augmentation and Ensemble Predictions to Prevent Machine-Learning Model Overfitting

Deep learning predictive models have the potential to simplify and automate medical imaging diagnostics by lowering the skill threshold for image interpretation. However, this requires predictive models that are generalized to handle subject variability as seen clinically. Here, we highlight methods to improve test accuracy of an image classifier model for shrapnel identification using tissue phantom image sets. Using a previously developed image classifier neural network—termed ShrapML—blind test accuracy was less than 70% and was variable depending on the training/test data setup, as determined by a leave one subject out (LOSO) holdout methodology. Introduction of affine transformations for image augmentation or MixUp methodologies to generate additional training sets improved model performance and overall accuracy improved to 75%. Further improvements were made by aggregating predictions across five LOSO holdouts. This was done by bagging confidences or predictions from all LOSOs or the top-3 LOSO confidence models for each image prediction. Top-3 LOSO confidence bagging performed best, with test accuracy improved to greater than 85% accuracy for two different blind tissue phantoms. This was confirmed by gradient-weighted class activation mapping to highlight that the image classifier was tracking shrapnel in the image sets. Overall, data augmentation and ensemble prediction approaches were suitable for creating more generalized predictive models for ultrasound image analysis, a critical step for real-time diagnostic deployment.

60 APPLIED LIFE SCIENCES↗

NASA NTRS2009

Validation of Community Models: 2. Development of a Baseline, Using the Wang-Sheeley-Arge Model

This paper is the second in a series providing independent validation of community models of the outer corona and inner heliosphere. Here I present a comprehensive validation of the Wang-Sheeley-Arge (WSA) model. These results will serve as a baseline against which to compare the next generation of comparable forecasting models. The WSA model is used by a number of agencies to predict Solar wind conditions at Earth up to 4 days into the future. Given its importance to both the research and forecasting communities, it is essential that its performance be measured systematically and independently. I offer just such an independent and systematic validation. I report skill scores for the model's predictions of wind speed and interplanetary magnetic field (IMF) polarity for a large set of Carrington rotations. The model was run in all its routinely used configurations. It ingests synoptic line of sight magnetograms. For this study I generated model results for monthly magnetograms from multiple observatories, spanning the Carrington rotation range from 1650 to 2074. I compare the influence of the different magnetogram sources and performance at quiet and active times. I also consider the ability of the WSA model to forecast both sharp transitions in wind speed from slow to fast wind and reversals in the polarity of the radial component of the IMF. These results will serve as a baseline against which to compare future versions of the model as well as the current and future generation of magnetohydrodynamic models under development for forecasting use.

MacNeice, Peter↗

NASA NTRS2015

Improved Rainfall Estimates and Predictions for 21st Century Drought Early Warning

As temperatures increase, the onset and severity of droughts is likely to become more intense. Improved tools for understanding, monitoring and predicting droughts will be a key component of 21st century climate adaption. The best drought monitoring systems will bring together accurate precipitation estimates with skillful climate and weather forecasts. Such systems combine the predictive power inherent in the current land surface state with the predictive power inherent in low frequency ocean-atmosphere dynamics. To this end, researchers at the Climate Hazards Group (CHG), in collaboration with partners at the USGS and NASA, have developed i) a long (1981-present) quasi-global (50degS-50degN, 180degW-180degE) high resolution (0.05deg) homogenous precipitation data set designed specifically for drought monitoring, ii) tools for understanding and predicting East African boreal spring droughts, and iii) an integrated land surface modeling (LSM) system that combines rainfall observations and predictions to provide effective drought early warning. This talk briefly describes these three components. Component 1: CHIRPS The Climate Hazards group InfraRed Precipitation with Stations (CHIRPS), blends station data with geostationary satellite observations to provide global near real time daily, pentadal and monthly precipitation estimates. We describe the CHIRPS algorithm and compare CHIRPS and other estimates to validation data. The CHIRPS is shown to have high correlation, low systematic errors (bias) and low mean absolute errors. Component 2: Hybrid statistical-dynamic forecast strategies East African droughts have increased in frequency, but become more predictable as Indo- Pacific SST gradients and Walker circulation disruptions intensify. We describe hybrid statistical-dynamic forecast strategies that are far superior to the raw output of coupled forecast models. These forecasts can be translated into probabilities that can be used to generate bootstrapped ensembles describing future climate conditions. Component 3: Assimilation using LSMs CHIRPS rainfall observations (component 1) and bootstrapped forecast ensembles (component 2) can be combined using LSMs to predict soil moisture deficits. We evaluate the skill such a system in East Africa, and demonstrate results for 2013.

Funk, Chris↗

NASA NTRS2015

Improved Rainfall Estimates and Predictions for 21st Century Drought Early Warning

Funk, Chris↗

NASA NTRS2008

Systematic Independent Validation of Inner Heliospheric Models

This presentation is the first in a series which will provide independent validation of community models of the outer corona and inner heliosphere. In this work we establish a set of measures to be used in validating this group of models. We use these procedures to generate a comprehensive set of results from the Wang- Sheeley-Arge (WSA) model which will be used as a baseline, or reference, against which to compare all other models. We also run a test of the validation procedures by applying them to a small set of results produced by the ENLIL Magnetohydrodynamic (MHD) model. In future presentations we will validate other models currently hosted by the Community Coordinated Modeling Center(CCMC), including a comprehensive validation of the ENLIL model. The Wang-Sheeley-Arge (WSA) model is widely used to model the Solar wind, and is used by a number of agencies to predict Solar wind conditions at Earth as much as four days into the future. Because it is so important to both the research and space weather forecasting communities, it is essential that its performance be measured systematically, and independently. In this paper we offer just such an independent and systematic validation. We report skill scores for the model's predictions of wind speed and IMF polarity for a large set of Carrington rotations. The model was run in all its routinely used configurations. It ingests line of sight magnetograms. For this study we generated model results for monthly magnetograms from the National Solar Observatory (SOLIS), Mount Wilson Observatory and the GONG network, spanning the Carrington rotation range from 1650 to 2068. We compare the influence of the different magnetogram sources, performance at quiet and active times, and estimate the effect of different empirical wind speed tunings. We also consider the ability of the WSA model to identify sharp transitions in wind speed from slow to fast wind. These results will serve as a baseline against which to compare future versions of the model

MacNeice, P. J.↗

NASA NTRS1992

Adaptive filtering and prediction of the Southern Oscillation index

Singular spectrum analysis (SSA), a variant of principal component analysis, is applied to a time series of the Southern Oscillation index (SOI). The analysis filters out variability unrelated to the Southern Oscillation and separates the high-frequency, 2- to 3-year variability, including the quasi-biennial oscillation, from the lower-frequency 4- to 6-year El Nino cycle. The maximum entropy method (MEM) is applied to forecasting the prefiltered SOI. Prediction based on MEM-associated autoregresive models has useful skill for 30-36 months. A 1993-1994 La Nina event is predicted based on data through February 1992.

Keppenne, Christian L.↗

NASA NTRS2017

CEDAR-GEM Challenge for Systematic Assessment of Ionosphere/Thermosphere Models in Predicting TEC During the 2006 December Storm Event

In order to assess current modeling capability of reproducing storm impacts on total electron content (TEC), we considered quantities such as TEC, TEC changes compared to quiet time values, and the maximum value of the TEC and TEC changes during a storm. We compared the quantities obtained from ionospheric models against ground-based GPS TEC measurements during the 2006 AGU storm event (14-15 December 2006) in the selected eight longitude sectors. We used 15 simulations obtained from eight ionospheric models, including empirical, physics-based, coupled ionosphere-thermosphere, and data assimilation models. To quantitatively evaluate performance of the models in TEC prediction during the storm, we calculated skill scores such as RMS error, Normalized RMS error (NRMSE), ratio of the modeled to observed maximum increase (Yield), and the difference between the modeled peak time and observed peak time. Furthermore, to investigate latitudinal dependence of the performance of the models, the skill scores were calculated for five latitude regions. Our study shows that RMSE of TEC and TEC changes of the model simulations range from about 3 TECU (total electron content unit, 1 TECU = 1016 el m2) (in high latitudes) to about 13 TECU (in low latitudes), which is larger than latitudinal average GPS TEC error of about 2 TECU. Most model simulations predict TEC better than TEC changes in terms of NRMSE and the difference in peak time, while the opposite holds true in terms of Yield. Model performance strongly depends on the quantities considered, the type of metrics used, and the latitude considered.

Shim, J. S.↗

NASA NTRSDate not supplied

Supply and Resource Management to Progressively Enable EIMO

BACKGROUND: Current medical operations in Low Earth Orbit (LEO) allow for real-time audio-video communication with a flight surgeon at mission control, resupply, and medical evacuation to earth on the order of hours. As mission profiles change from LEO to the Moon, Mars, and beyond, medical risk as a contribution to overall mission risk is anticipated to rise substantially. Concurrently, due to the distance, the medical systems on board vehicles proposed for these mission types are expected to have reduced mass and volume allocation. Together, astronaut crews will be at a higher risk of major medical events, be required to perform a broader set of tasks, and have substantially reduced resources and support to do so. Earth Independent Medical Operations (EIMO) aims to identify and fill the gaps present in this progressively changing paradigm. OVERVIEW: Medical supplies, resources, and skills are central to spaceflight medical systems. Vehicles used for non-LEO missions are anticipated to be smaller and thus the medical system will also need to have reduced mass, volume, and power. Medical resources are another form of consumable and may need resupply or pre-deployment to meet crew needs. One particular concern is the degradation of medications which become less efficacious and potentially toxic with time, particularly given environmental conditions such as temperature, humidity, oxygen, and radiation which have not yet been fully studied. Supply and resource management in LEO is dependent on resupply, however the supply chain of transporting equipment does not yet have a clear infrastructure for missions beyond LEO. DISCUSSION: EIMO is intended to systemically identify and fill these gaps with forward-looking solutions. In mission monitoring of resources with technology like RFID, improving medical resource longevity, targeted resupply and careful pre-mission planning will be central to facilitate crew health and performance. One approach to optimizing resources is the Informing Mission Planning via Analysis of Complex Tradespaces (IMPACT) tool, which uses probabilistic risk assessment (PRA) to quantitatively predict medical risk and identify resources and skills that mitigate this risk. This evidence based, quantitative analysis prediction tool and several other approaches to meeting the challenge of supply and resource management are discussed.

Arian Anderson↗

NASA NTRS1984

Theoretical Modeling for Numerical Weather Prediction

The goal is to utilize predictability theory and numerical experimentation to identify and understand some of the dynamical processes which must be modeled more realistically if large-scale numerical weather predictions are to be improved. The emphasis is on the use of relatively simple models to exlore the properties of physically comprehensive general circulation models (GCM's). A global linear quasi-geostrophic model and the Goddard Laboratory for Atmospheric Sciences (GLAS) GCM were used to investigate several mechanisms which are responsible for the decay of large-scale forecast skill in mid-latitude numerical weather predictions. Five-day forecasts for an ensemble of cases were made using First GARP Global Experiment data. It was found that forecast skill depends crucially on the specification of the stationary forcing. A lack of stationary forcing leads to spurious westwad propagation of the ultralong waves. Forecasts made with stationary forcings derived from climatological data are superior to those using forcings inferred from observations immediately preceding the forecast period. Interhemispheric forecast differences were analyzed, and the model errors were compared to errors of a simple persistence-damped-to-climatology scheme and to errors of the GLAS GCM.

Somerville, R. C. J.↗

NASA NTRS2012

Assessment of the APCC Coupled MME Suite in Predicting the Distinctive Climate Impacts of Two Flavors of ENSO during Boreal Winter

Forecast skill of the APEC Climate Center (APCC) Multi-Model Ensemble (MME) seasonal forecast system in predicting two main types of El Nino-Southern Oscillation (ENSO), namely canonical (or cold tongue) and Modoki ENSO, and their regional climate impacts is assessed for boreal winter. The APCC MME is constructed by simple composite of ensemble forecasts from five independent coupled ocean-atmosphere climate models. Based on a hindcast set targeting boreal winter prediction for the period 19822004, we show that the MME can predict and discern the important differences in the patterns of tropical Pacific sea surface temperature anomaly between the canonical and Modoki ENSO one and four month ahead. Importantly, the four month lead MME beats the persistent forecast. The MME reasonably predicts the distinct impacts of the canonical ENSO, including the strong winter monsoon rainfall over East Asia, the below normal rainfall and above normal temperature over Australia, the anomalously wet conditions across the south and cold conditions over the whole area of USA, and the anomalously dry conditions over South America. However, there are some limitations in capturing its regional impacts, especially, over Australasia and tropical South America at a lead time of one and four months. Nonetheless, forecast skills for rainfall and temperature over East Asia and North America during ENSO Modoki are comparable to or slightly higher than those during canonical ENSO events.

Jeong, Hye-In↗

NASA NTRS1974

Feedback precision and postfeedback interval duration

Precision of feedback gain was manipulated in a simple positioning task. An optimum was found; an increase in precision past that optimum produced deleterious effects upon rate of acquisition. In a second study, increasing postfeedback interval removed that optimum. The feedback precision effects were then replicated in a timing task. The combined results of the 3 studies were interpreted as supportive of an information-processing approach to the study of postfeedback interval events for simple motor skills. The findings additionally supported specific predictions by Bilodeau and deductions from Adams' 1971 theory of motor learning.

Rogers, C. A., Jr.↗

NASA NTRS1986

Progress in mental workload measurement

Two new techniques are described, one using subjective, the other physiological data for the measurement of workload in complex tasks. The subjective approach uses fuzzy measurement to analyze and predict the difficulty of combinations of skill based and rule based behavior from the difficulty of skill based behavior and rule based behavior measured separately. The physiological technique offers an on-line real-time filter for measuring the Mulder signal at 0.1 Hz in the heart rate variability spectrum.

Moray, Neville↗

NASA NTRS2013

SMAP Radar Processing and Calibration

The Soil Moisture Active Passive (SMAP) mission is part of the NASA space-based Earth observation program, and consists of an L-band radar and radiometer scheduled for launch into sun synchronous orbit in late 2014. A joint effort of the Jet Propulsion Laboratory (JPL) and the Goddard Space Flight Center (GSFC), the SMAP mission draws heavily on the design and risk reduction heritage of the Hydrosphere State (Hydros) mission [1], [2]. The SMAP science and applications objectives are to: 1) understand processes that link the terrestrial water, energy and carbon cycles, 2) estimate global water and energy fluxes at the land surface, 3) quantify net carbon flux in boreal landscapes, 4) enhance weather and climate forecast skill, and 5) develop improved flood prediction and drought monitoring capability. To meet these science objectives, SMAP ground processing will combine the attributes of the radar and radiometer observations (in terms of their spatial resolution and sensitivity to soil moisture, surface roughness, and vegetation) to estimate soil moisture with 4% volumetric accuracy at a resolution of 10 km, and freeze-thaw state at a resolution of 1-3 km. Model sensitivities translate the soil moisture accuracy to a radar backscatter accuracy of 1 dB (1 sigma) at 3 km resolution and a brightness temperature accuracy of 1.3 K at 40 km resolution. This paper will describe the level 1 radar processing and calibration challenges and the choices made so far for the algorithms and software implementation.

Synthetic Aperture Radar (SAR),↗