Search NASA⌕ Search

SEARCH · Search NASA

Results for “Prediction skill”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

Evaluation of FluSight influenza forecasting in the 2021–22 and 2022–23 seasons with a new target laboratory-confirmed influenza hospitalizations

Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. For the 2021–22 and 2022–23 influenza seasons, 26 forecasting teams provided national and jurisdiction-specific probabilistic predictions of weekly confirmed influenza hospital admissions for one-to-four weeks ahead. Forecast skill is evaluated using the Weighted Interval Score (WIS), relative WIS, and coverage. Six out of 23 models outperform the baseline model across forecast weeks and locations in 2021–22 and 12 out of 18 models in 2022–23. Averaging across all forecast targets, the FluSight ensemble is the 2nd most accurate model measured by WIS in 2021–22 and the 5th most accurate in the 2022–23 season. Forecast skill and 95% coverage for the FluSight ensemble and most component models degrade over longer forecast horizons. In this work we demonstrate that while the FluSight ensemble was a robust predictor, even ensembles face challenges during periods of rapid change.

59 BASIC BIOLOGICAL SCIENCES↗

Forecasting atmospheric angular momentum and length-of-day using operational meteorological models

Forecasts of zonal wind fields produced by the medium-range forecast model of the U.S. National Meteorological Center are used to create predictions of the atmosphere's angular momentum at lead times of 1-10 days. The skill of these forecasts, which are of interest to those concerned with monitoring changes in the length-of-day for navigational purposes, is assessed, and the regions in the atmosphere that contribute most importantly to forecast errors are identified.

Rosen, R. D.↗

Development, Validation, and Application of OSSEs at NASA-GMAO

The GMAO OSSE framework has two general classes of applications. One is to estimate the potential improvements to weather forecasting and analysis by using new proposed instruments that are not as yet built or deployed. This exploits the simulated nature of the OSSE. The other is to assess various aspects of the GMAO data assimilation system. This exploits the availability of truth provided by the OSSE. Two examples of the first class of application will be offered. One concerns deployment of constellations of passive MW sounders on small CUBESATs placed in very low orbits. The other concern is increasing the frequency of radiosonde observations to 4-times daily at all current stations. Several examples of the second class will also be presented. One is an estimation of analysis error characteristics. Another is a comparison between covariances directly determined from explicitly known background errors and those estimated by computing differences between lagged forecasts using the NMC-method prior to tuning. A third is a comparison of effects of model and observation errors on analysis and forecast skill. The last is an examination of spectra of forecast errors in a study of predictability. This latter is an ongoing study.

OSSE↗

Bayesian Safety Risk Modeling of Human-Flightdeck Automation Interaction

Usage of automatic systems in airliners has increased fuel efficiency, added extra capabilities, enhanced safety and reliability, as well as provide improved passenger comfort since its introduction in the late 80's. However, original automation benefits, including reduced flight crew workload, human errors or training requirements, were not achieved as originally expected. Instead, automation introduced new failure modes, redistributed, and sometimes increased workload, brought in new cognitive and attention demands, and increased training requirements. Modern airliners have numerous flight modes, providing more flexibility (and inherently more complexity) to the flight crew. However, the price to pay for the increased flexibility is the need for increased mode awareness, as well as the need to supervise, understand, and predict automated system behavior. Also, over-reliance on automation is linked to manual flight skill degradation and complacency in commercial pilots. As a result, recent accidents involving human errors are often caused by the interactions between humans and the automated systems (e.g., the breakdown in man-machine coordination), deteriorated manual flying skills, and/or loss of situational awareness due to heavy dependence on automated systems. This paper describes the development of the increased complexity and reliance on automation baseline model, named FLAP for FLightdeck Automation Problems. The model development process starts with a comprehensive literature review followed by the construction of a framework comprised of high-level causal factors leading to an automation-related flight anomaly. The framework was then converted into a Bayesian Belief Network (BBN) using the Hugin Software v7.8. The effects of automation on flight crew are incorporated into the model, including flight skill degradation, increased cognitive demand and training requirements along with their interactions. Besides flight crew deficiencies, automation system failures and anomalies of avionic systems are also incorporated. The resultant model helps simulate the emergence of automation-related issues in today's modern airliners from a top-down, generalized approach, which serves as a platform to evaluate NASA developed technologies

Ancel, Ersin↗

The Impact of Ocean Observations in Seasonal Climate Prediction

The ocean provides the most significant memory for the climate system. Hence, a critical element in climate forecasting with coupled models is the initialization of the ocean with states from an ocean data assimilation system. Remotely-sensed ocean surface fields (e.g., sea surface topography, SST, winds) are now available for extensive periods and have been used to constrain ocean models to provide a record of climate variations. Since the ocean is virtually opaque to electromagnetic radiation, the assimilation of these satellite data is essential to extracting the maximum information content. More recently, the Argo drifters have provided unprecedented sampling of the subsurface temperature and salinity. Although the duration of this observation set has been too short to provide solid statistical evidence of its impact, there are indications that Argo improves the forecast skill of coupled systems. This presentation will address the impact these different observations have had on seasonal climate predictions with the GMAO's coupled model.

Rienecker, Michele↗

Large Ensemble Particle Filter for Spatial Climate Reconstructions Using a Linear Inverse Model

Proxy records that document the last 2000 years of climate provide evidence for the wide range of the natural climate variability from inter-annual to secular timescales not captured by the short window of recent direct observations. Assessing climate models ability to reproduce such natural variations is crucial to understand climate sensitivity and impacts of future climate change. Paleoclimate data assimilation (PDA) offers a powerful way to extend the short instrumental period by optimally combining the physics described by General Circulation Climate Models (GCMs) with information from available proxy records while taking into account their uncertainties. Here we present a new PDA approach based on a sequential importance resampling (SIR) Particle filter (PF) that uses Linear Inverse Modeling (LIM) as an emulator of several CMIP-class GCMs. We examine in a perfect-model framework the skill of the various LIMs to forecast the dynamics of the surface temperatures and provide spatial field reconstructions over the last millennium in a SIR PF. Our results show that the LIMs allow for skillful ensemble forecasts at 1-year lead-time based on GCMs dynamical knowledge with best prediction in the tropics and the North Atlantic. The PDA further provides a set of physically consistent spatial fields allowing robust uncertainty quantification related to climate models biases and proxy spatial sampling. Our results indicate that the LIM yields dynamical memory improving climate variability reconstructions and support the use of the LIM as a GCM-emulator in real reconstruction to propagate large ensembles of particles at low cost in SIR PF.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of Leading Modes of Climate Variability in the CMIP Archives

The adequate simulation of internal climate variability is key for our understanding of climate as it underpins efforts to attribute historical events, predict on seasonal and decadal time scales, and isolate the effects of climate change. Here the skill of models in reproducing observed modes of climate variability is assessed, both across and within the CMIP3, CMIP5, and CMIP6 archives, in order to document model capabilities, progress across ensembles, and persisting biases. A focus is given to the well-observed tropical and extratropical modes that exhibit small intrinsic variability relative to model structural uncertainty. These include El Niño–Southern Oscillation (ENSO), the Pacific decadal oscillation (PDO), the North Atlantic Oscillation (NAO), and the northern and southern annular modes (NAM and SAM). Significant improvements are identified in models’ representation of many modes. Canonical biases, which involve both amplitudes and patterns, are generally reduced across model generations. For example, biases in ENSO-related equatorial Pacific sea surface temperature, which extend too far westward, and associated atmospheric teleconnections, which are too weak, are reduced. Stronger tropical expression of the PDO in successive CMIP generations has characterized their improvement, with some CMIP6 models generating patterns that lie within the range of observed estimates. For the NAO, NAM, and SAM, pattern correlations with observations are generally higher than for other modes and slight improvements are identified across successive model generations. Finally, for ENSO and PDO spectra and extratropical modes, changes are small compared to internal variability, precluding definitive statements regarding improvement.

54 ENVIRONMENTAL SCIENCES↗

Kennedy's Biomedical Laboratory Makes Multi-Tasking Look Easy

If it is one thing that Florida has in abundance, it is sunshine and with that sunshine heat and humidity. For workers at the Kennedy Space Center that have to work outside in the heat and humidity, heat exhaustion/stroke is a real possibility. It might help people to know that Kennedy's Biomedical Laboratory has been testing some new Koolvests(Trademark) that can be worn underneath SCAPE suits. They have also been working on how to block out high noise levels; in fact, Don Doerr, chief of the Biomedical Lab, says, "The most enjoyable aspect is knowing that the Biomedical Lab and the skills of its employees have been used to support safe space flight, not only for the astronaut flight crew, but just as important for the ground processing personnel as well." The NASA Biomedical Laboratory has existed in the John F. Kennedy's Operations and Checkout Building since the Apollo Program. The primary mission of this laboratory has been the biomedical support to major, manned space programs that have included Apollo, Apollo-Soyuz, Skylab, and Shuttle. In this mission, the laboratory has been responsible in accomplishing much of the technical design, planning, provision, fabrication, and maintenance of flight and ground biomedical monitoring instrumentation. This includes the electronics in the launch flight suit and similar instrumentation systems in the spacecraft. (Note: The Lab checked out the system for STS-128 at Pad A using Firing room 4 and ground support equipment in the lab.) During Apollo, there were six engineers and ten technicians in the facility. This has evolved today to two NASA engineers and two NASA technicians, a Life Science Support contract physiologist and part-time support from an LSSC nurse and physician. Over the years, the lab has enjoyed collaboration with outside agencies and investigators. These have included on-site support to the Ames Research Center bed rest studies (seven years) and the European Space Agency studies in Toulouse, France (two years). The lab has also actively collaborated with the US Army Institute for Surgical Research, the USAF School of Aerospace Medicine, and the USN Naval Experimental Diving Unit. Because the lab often evaluates various forms of commercial-off-the-shelf life support equipment, the laboratory works closely with private companies, both domestic and foreign. The European companies seem to be more proactive and participatory with the advancement of personal protective equipment. Because these companies have viewed the space program's unique need for advanced forms of personal protective equipment, some have responded with new designs based on the prediction that these advances will soon find markets in the commercial sector. Using much of the same skills and equipment, the laboratory also addresses physiological testing of humans by supporting flight experiments and personnel involved with ground processing. While Johnson Space Center is primarily responsible for flight experiments, the Kennedy's Biomedical Lab provides the local support. However, as stated above, there are many challenges facing KSC workers that gain the attention of this lab in the measurement of the problem and the selection and testing of countermeasures. These include respiratory protection, whole body suits, hearing protection and heat stress, among many others.

Dunn, Carol Anne↗

Challenges of COVID-19 Case Forecasting in the US, 2020–2021

During the COVID-19 pandemic, forecasting COVID-19 trends to support planning and response was a priority for scientists and decision makers alike. In the United States, COVID-19 forecasting was coordinated by a large group of universities, companies, and government entities led by the Centers for Disease Control and Prevention and the US COVID-19 Forecast Hub ( https://covid19forecasthub.org ). We evaluated approximately 9.7 million forecasts of weekly state-level COVID-19 cases for predictions 1–4 weeks into the future submitted by 24 teams from August 2020 to December 2021. We assessed coverage of central prediction intervals and weighted interval scores (WIS), adjusting for missing forecasts relative to a baseline forecast, and used a Gaussian generalized estimating equation (GEE) model to evaluate differences in skill across epidemic phases that were defined by the effective reproduction number. Overall, we found high variation in skill across individual models, with ensemble-based forecasts outperforming other approaches. Forecast skill relative to the baseline was generally higher for larger jurisdictions (e.g., states compared to counties). Over time, forecasts generally performed worst in periods of rapid changes in reported cases (either in increasing or decreasing epidemic phases) with 95% prediction interval coverage dropping below 50% during the growth phases of the winter 2020, Delta, and Omicron waves. Ideally, case forecasts could serve as a leading indicator of changes in transmission dynamics. However, while most COVID-19 case forecasts outperformed a naïve baseline model, even the most accurate case forecasts were unreliable in key phases. Further research could improve forecasts of leading indicators, like COVID-19 cases, by leveraging additional real-time data, addressing performance across phases, improving the characterization of forecast confidence, and ensuring that forecasts were coherent across spatial scales. In the meantime, it is critical for forecast users to appreciate current limitations and use a broad set of indicators to inform pandemic-related decision making.

59 BASIC BIOLOGICAL SCIENCES↗

Computational Modeling of Atmospheric Processes at Texas Southern University

Texas Southern University (TSU) is strengthening its research program in atmospheric chemistry and physics with a climate science emphasis by leveraging partnerships with the U.S. Department of Energy’s Atmospheric Radiation Measurement (ARM) Facility, Brookhaven National Laboratory (BNL), and the Tracking Aerosol Convection Interactions ExpeRiment (TRACER). This RDPP-supported program focuses on secondary organic aerosols (SOAs) and reactive atmospheric species that influence cloud formation, precipitation processes, and radiative forcing. SOAs play a critical role in cloud microphysics and Earth’s energy balance, yet the chemical and physical mechanisms governing SOA–cloud interactions remain a significant source of uncertainty in predictive climate models. Through computational modeling, observational data analysis, and national laboratory collaboration, this program develops a skilled cohort of students trained in atmospheric science, environmental data analysis, and climate-relevant modeling. These research experiences build technical competencies that are transferable to careers in government laboratories, academia, and industry. By engaging students from historically underrepresented communities in high-impact climate research, TSU expands participation in the atmospheric sciences workforce while contributing meaningful scientific insights to DOE-supported ARM research activities. This partnership strengthens national capacity in climate science and supports the development of the next generation of atmospheric researchers.

54 ENVIRONMENTAL SCIENCES↗

Flood and Landslide Applications of High Time Resolution Satellite Rain Products

Experimental, potentially real-time systems to detect floods and landslides related to heavy rain events are described. A key basis for these applications is high time resolution satellite rainfall analyses. Rainfall is the primary cause for devastating floods across the world. However, in many countries, satellite-based precipitation estimation may be the best source of rainfall data due to insufficient ground networks and absence of data sharing along many trans-boundary river basins. Remotely sensed precipitation from the NASA's TRMM Multi-satellite Precipitation Analysis (TMPA) operational system (near real-time precipitation at a spatial-temporal resolution of 3 hours and 0.25deg x 0.25deg) is used to monitor extreme precipitation events. Then these data are ingested into a macro-scale hydrological model which is parameterized using spatially distributed elevation, soil and land cover datasets available globally from satellite remote sensing. Preliminary flood results appear reasonable in terms of location and frequency of events, with implementation on a quasi-global basis underway. With the availability of satellite rainfall analyses at fine time resolution, it has also become possible to assess landslide risk on a near-global basis. Early results show that landslide occurrence is closely associated with the spatial patterns and temporal distribution of TRMM rainfall characteristics. Particularly, the number of landslides triggered by rainfall is related to rainfall climatology, antecedent rainfall accumulation, and intensity-duration of rainstorms. For the purpose of prediction, an empirical TMPA-based rainfall intensity-duration threshold is developed and shown to have skill in determining potential areas of landslides. These experimental findings, in combination with landslide surface susceptibility information based on satellite-based land surface information, form a starting point towards a potential operational landslide monitoring/warning system around the globe.

Adler, Robert F.↗

Metrics for Linear Kinematic Features in Sea Ice

The treatment of leads as cracks or discontinuities (see Coon et al. presentation) requires some shift in the procedure of evaluation and comparison of lead-resolving models and their validation against observations. Common metrics used to evaluate ice model skills are by and large an adaptation of a least square "metric" adopted from operational numerical weather prediction data assimilation systems and are most appropriate for continuous fields and Eilerian systems where the observations and predictions are commensurate. However, this class of metrics suffers from some flaws in areas of sharp gradients and discontinuities (e.g., leads) and when Lagrangian treatments are more natural. After a brief review of these metrics and their performance in areas of sharp gradients, we present two new metrics specifically designed to measure model accuracy in representing linear features (e.g., leads). The indices developed circumvent the requirement that both the observations and model variables be commensurate (i.e., measured with the same units) by considering the frequencies of the features of interest/importance. We illustrate the metrics by scoring several hypothetical "simulated" discontinuity fields against the lead interpreted from RGPS observations.

Levy, G.↗

The Boreal Winter El Niño Precipitation Response over North America: Insights into Why January is More Difficult to Predict than February

This study examines the within−season monthly variation of the El Niño response over North America during December−March using the NASA/GEOS model. In agreement with previous studies, the skill of 1−month lead GEOS coupled model forecasts of precipitation over North America is largest (smallest) for February (January), with similar results in uncoupled mode. A key finding is that the relatively poor January skillis the result ofthe model placing the main circulation anomaly over the northeast Pacific slightly to the west of the observed, resulting inprecipitation anomalies that lie off the coast instead of over land as observed. In contrast, during February the observed circulation anomaly over the northeast Pacific shifts westward, lining up with the predicted anomaly which is essentially unchanged from January, resulting in both the observed and predicted precipitation anomalies remaining off the coast. Furthermore, the largest precipitation anomalies occur along the southern tier of states associated with an eastward extended jet–something that the models capture reasonably well. Simulations with a stationary wave model indicate that the placement of January El Niño response to the west of the observed over the northeast Pacificis the result of biases in the January climatological stationary waves, rather than errors in the tropical Pacific El Niño heating anomalies in January. Furthermore, evidence is provided that the relatively poor simulation of the observed January climatology, characterized by a strengthened North Pacific jet and enhanced ridge over western North America, can be traced back to biases in the January climatology heating over the Tibet region and the tropical western Pacific.

Young-kwon Lim↗

Impasse-driven tutoring for reactive skill acquisition

We are interested in developing effective performance-oriented training for the operation of systems that are used for monitor and control purposes. We have focused on one such system, the communications Link Monitor and Control (LMC) system used in NASA's Deep Space Network (DSN), which is a worldwide system for navigating, tracking and communicating with unmanned interplanetary spacecraft. The tasks in this domain are procedural in nature and require reactive, goal-oriented skills; we have previously described a cognitive model for problem solving that accounts for both novice and expert levels of behavior as well as how skill is acquired. Our cognitive modeling work in this task domain led us to make a number of predictions about tutoring that have influenced the design of the system described in this paper.

Hill, Randall W., Jr.↗

Results From The First Interactive Scientific Expert System In Space

We describe recent work on automation and support of science experiments on the Space Shuttle. PI-in-a-Box, the first interactive expert system for scientific support, was flown onboard STS-58 in October 1993. It was designed to help astronauts perform better science in space by providing expert-level advice in real-time. The system provides help with 'technician' tasks such as data acquisition, data quality monitoring, and diagnostic, troubleshooting and repair. The system also performs 'expert' reasoning on the scientific model and experiment steps scheduling. It can make scheduling recommendations based on hardware status, time available, and agreement between predictions and flight data. The system was successfully deployed and used throughout the mission. Most of the 'technician' skills of the system were used with success. The scientific model study performed correctly, but we believe the results were not exploited by the astronauts because of Spacelab operational pressures. Also, portions of the system were designed around rigid schedule constraints that were essentially relaxed during the mission. This version was tailored to a specific vestibular experiment known as the 'rotating dome'. We are currently expanding our effort in various scientific (other life and microgravity sciences experiments) and technological directions (integrated telescience, astronaut science advisor, and autonomous control).

Groleau, Nicolas↗

Results from the First Interactive Scientific Expert System in Space

We describe recent work on automation and support of science experiments on the Space Shuttle. PI-in-a-Box, the first interactive expert system for scientific support, was flown onboard STS-58 in October 1993. It was designed to help astronauts perform better science in space by providing expert-level advice in real-time. The system provides help with "technician" tasks such as data acquisition, data quality monitoring, and diagnostic, troubleshooting and repair. The system also performs "expert" reasoning on the scientific model and experiment steps scheduling. It can make scheduling recommendations based on hardware status, time available, and agreement between predictions and flight data. The system was successfully deployed and used throughout the mission. Most of the "technician" skills of the system were used with success. The scientific model study performed correctly, but we believe the results were not exploited by the astronauts because of Spacelab operational pressures. Also, portions of the system were designed around rigid schedule constraints that were essentially relaxed during the mission. This version was tailored to a specific vestibular experiment known as the "rotating dome". We are currently expanding our effort in various scientific (other life and microgravity sciences experiments) and technological directions (integrated telescience, astronaut science advisor, and autonomous control).

Groleau, Nicolas↗

Simulating the Effects of Irrigation over the U.S. in a Land Surface Model Based on Satellite Derived Agricultural Data

A novel method is introduced for integrating satellite derived irrigation data and high-resolution crop type information into a land surface model (LSM). The objective is to improve the simulation of land surface states and fluxes through better representation of agricultural land use. Ultimately, this scheme could enable numerical weather prediction (NWP) models to capture land-atmosphere feedbacks in managed lands more accurately and thus improve forecast skill. Here we show that application of the new irrigation scheme over the continental US significantly influences the surface water and energy balances by modulating the partitioning of water between the surface and the atmosphere. In our experiment, irrigation caused a 12% increase in evapotranspiration (QLE) and an equivalent reduction in the sensible heat flux (QH) averaged over all irrigated areas in the continental US during the 2003 growing season. Local effects were more extreme: irrigation shifted more than 100 W/m from QH to QLE in many locations in California, eastern Idaho, southern Washington, and southern Colorado during peak crop growth. In these cases, the changes in ground heat flux (QG), net radiation (RNET), evapotranspiration (ET), runoff (R), and soil moisture (SM) were more than 3 W/m(sup 2), 20 W/m(sup 2), 5 mm/day, 0.3 mm/day, and 100 mm, respectively. These results are highly relevant to continental- to global-scale water and energy cycle studies that, to date, have struggled to quantify the effects of agricultural management practices such as irrigation. Based on the results presented here, we expect that better representation of managed lands will lead to improved weather and climate forecasting skill when the new irrigation scheme is incorporated into NWP models such as NOAA's Global Forecast System (GFS).

Ozdogan, Mutlu↗