Search NASA⌕ Search

SEARCH · Search NASA

Results for “forecast error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41

Gaussian Process for Flight Delay Prediction: Learning a Stochastic Process

This paper presents a machine-learning approach to predict flight delays. Whereas neural networks are extensively studied for predictive capabilities, they involve non-intuitive design and extensive analysis, particularly in training and optimization processes. Instead, the proposed framework employs Gaussian Processes as a supervised learning technique for flight delay prediction. This data-driven approach trains the model using prior information, specifically the mean and covariance tied to existing data. The proposed Gaussian Process Regression (GPR) model employs the day of flight as a pivotal feature for delay forecasting. We analyze flights from various routes and gauge the accuracy of the presented learning technique by comparing the predicted delays with the actual ones. Given the inherent challenges in precisely forecasting delays, we predict the delays with a 95 % confidence interval. Also, an error propagation analysis in the prediction horizon is carried out to determine the optimal time frame for prediction. The proposed method for flight delay prediction is important as airlines can strategize flight operations and issue timely advisories.

stochastic↗

Short-term wind forecasting using statistical models with a fully observable wind flow

The utility of model output data from the Weather Research and Forecasting mesoscale model is explored for very short-term forecasting (5-30 minutes horizon) of wind speed to be used in large scale simulations of an autonomous electric power grid. Using this synthetic data for the development and evaluation of short-term forecasting algorithms offer many unique advantages over observational data, such as the ability to observe the full wind flow field in the surrounding region. Several short-term forecasting algorithms are implemented and evaluated using the synthetic data at several different time horizons and for three different geographic locations. Comparison is made with observational data from one location. We find that short-term forecasts of the synthetic data considering wind flow from the surrounding region perform 26% better than persistence in terms of root mean square error at the 5-minute time horizon. This improvement is comparable to studies of observational data in the literature. These results provide motivation to use synthetic data for short term forecasting in grid simulations, and open the door to future algorithmic improvements.

17 WIND ENERGY↗

Quantifying Errors in 3D CME Parameters Derived from Synthetic Data Using White-Light Reconstruction Techniques

Current efforts in space weather forecasting of CMEs have been focused on predicting their arrival time and magnetic structure. To make these predictions, methods have been developed to derive the true CME speed, size, position, and mass, among others. Difficulties in determining the input parameters for CME forecasting models arise from the lack of direct measurements of the coronal magnetic fields and uncertainties in estimating the CME 3D geometric and kinematic parameters after eruption. White-light coronagraph images are usually employed by a variety of CME reconstruction techniques that assume more or less complex geometries. This is the first study from our International Space Science Institute (ISSI) team “Understanding Our Capabilities in Observing and Modeling Coronal Mass Ejections”, in which we explore how subjectivity affects the 3D CME parameters that are obtained from the Graduated Cylindrical Shell (GCS) reconstruction technique, which is widely used in CME research. To be able to quantify such uncertainties, the “true” values that are being fitted should be known, which are impossible to derive from observational data. We have designed two different synthetic scenarios where the “true” geometric parameters are known in order to quantify such uncertainties for the first time. We explore this by using two sets of synthetic data: 1) Using the ray-tracing option from the GCS model software itself, and 2) Using 3D magnetohydrodynamic (MHD) simulation data from the Magnetohydrodynamic Algorithm outside a Sphere code. Our experiment includes different viewing configurations using single and multiple viewpoints. CME reconstructions using a single viewpoint had the largest errors and error ranges overall for both synthetic GCS and simulated MHD white-light data. As the number of viewpoints increased from one to two, the errors decreased by approximately 4° in latitude, 22° in longitude, 14° in tilt, and 10° in half-angle. Our results quantitatively show the critical need for at least two viewpoints to be able to reduce the uncertainty in deriving CME parameters. We did not find a significant decrease in errors when going from two to three viewpoints for our specific hypothetical three spacecraft scenario using synthetic GCS white-light data. As we expected, considering all configurations and numbers of viewpoints, the mean absolute errors in the measured CME parameters are generally significantly higher in the case of the simulated MHD white-light data compared to those from the synthetic white-light images generated by the GCS model. We found the following CME parameter error bars as a starting point for quantifying the minimum error in CME parameters from white-light reconstructions: Δθ (latitude)=6° +2° -3° , Δϕ (longitude)=11° +18° -6° , Δγ (tilt)=25° +8° -7° , Δx (half-angle)=10° +12° -6° , Δh (height)=0.6 +1.2 -0.4 R ⨀ , and Δκ (ratio)=0.1 +0.03 -0.02 .

Coronal mass ejections↗

Quantifying Errors in 3D CME Parameters Derived From Synthetic Data Using White-Light Reconstruction Techniques

Current efforts in space weather forecasting of CMEs have been focused on predicting their arrival time and magnetic structure. To make these predictions, methods have been developed to derive the true CME speed, size, position, and mass, among others. Difficulties in determining the input parameters for CME forecasting models arise from the lack of direct measurements of the coronal magnetic fields and uncertainties in estimating the CME 3D geometric and kinematic parameters after eruption. White-light coronagraph images are usually employed by a variety of CME reconstruction techniques that assume more or less complex geometries. This is the first study from our International Space Science Institute (ISSI) team “Understanding Our Capabilities in Observing and Modeling Coronal Mass Ejections”, in which we explore how subjectivity affects the 3D CME parameters that are obtained from the Graduated Cylindrical Shell (GCS) reconstruction technique, which is widely used in CME research. To be able to quantify such uncertainties, the “true” values that are being fitted should be known, which are impossible to derive from observational data. We have designed two different synthetic scenarios where the “true” geometric parameters are known in order to quantify such uncertainties for the first time. We explore this by using two sets of synthetic data: 1) Using the ray-tracing option from the GCS model software itself, and 2) Using 3D magnetohydrodynamic (MHD) simulation data from the Magnetohydrodynamic Algorithm outside a Sphere code. Our experiment includes different viewing configurations using single and multiple viewpoints. CME reconstructions using a single viewpoint had the largest errors and error ranges overall for both synthetic GCS and simulated MHD white-light data. As the number of viewpoints increased from one to two, the errors decreased by approximately 4° in latitude, 22° in longitude, 14° in tilt, and 10° in half-angle. Our results quantitatively show the critical need for at least two viewpoints to be able to reduce the uncertainty in deriving CME parameters. We did not find a significant decrease in errors when going from two to three viewpoints for our specific hypothetical three spacecraft scenario using synthetic GCS white-light data. As we expected, considering all configurations and numbers of viewpoints, the mean absolute errors in the measured CME parameters are generally significantly higher in the case of the simulated MHD white-light data compared to those from the synthetic white-light images generated by the GCS model. We found the following CME parameter error bars as a starting point for quantifying the minimum error in CME parameters from white-light reconstructions: Δθ (latitude)=6° +2° -3° , Δϕ (longitude)=11° +18° -6° , Δγ (tilt)=25° +8° -7° , Δx (half-angle)=10° +12° -6° , Δh (height)=0.6 +1.2 -0.4 R ⨀ , and Δκ (ratio)=0.1 +0.03 -0.02 .

Coronal mass ejections↗

An Assessment of the Skill of GEOS-5 Seasonal Forecasts

The seasonal forecast skill of the NASA Global Modeling and Assimilation Office coupled global climate model (CGCM) is evaluated based on an ensemble of 9-month lead forecasts for the period 1993 to 2010. The results from the current version (V2) of the CGCM consisting of the GEOS-5 AGM coupled to the MOM4 ocean model are compared with those from an earlier version (V1) in which the AGCM (the NSIPP model) was coupled to the Poseidon Ocean Model. It was found that the correlation skill of the Sea Surface Temperature (SST) forecasts is generally better in V2, especially over the sub-tropical and tropical central and eastern Pacific, Atlantic, and Indian Ocean. Furthermore, the improvement in skill in V2 mainly comes from better forecasts of the developing phase of ENSO from boreal spring to summer. The skill of ENSO forecasts initiated during the boreal winter season, however, shows no improvement in terms of correlation skill, and is in fact slightly worse in terms of root mean square error (RMSE). The degradation of skill is found to be due to an excessive ENSO amplitude. For V1, the ENSO amplitude is too strong in forecasts starting in boreal spring and summer, which causes large RMSE in the forecast. For V2, the ENSO amplitude is slightly stronger than that in observations and V1 for forecasts starting in boreal winter season. An analysis of the terms in the SST tendency equation, shows that this is mainly due to an excessive zonal advective feedback. In addition, V2 forecasts that are initiated during boreal winter season, exhibit a slower phase transition of El Nino, which is consistent with larger amplitude of ENSO after the ENSO peak season. It is found that this is due to weak discharge of equatorial Warm Water Volume (WWV). In both observations and V1, the discharge of equatorial WWV leads the equatorial geostrophic easterly current so as to damp the El Nino starting in January. This process is delayed by about 2 months in V2 due to the slower phase transition of the equatorial zonal current from westerly to easterly.

GMAO↗

Forecasting the Arrival Time of Coronal Mass Ejections: Analysis of the CCMC CME Scoreboard

Accurate forecasting of the properties of coronal mass ejections (CMEs) as they approach Earth is now recognized as an important strategic objective for both NOAA and NASA. The time of arrival of such events is a key parameter, one that had been anticipated to be relatively straightforward to constrain. In this study, we analyze forecasts submitted to the Community Coordinated Modeling Center at NASA's Goddard Space Flight Center over the last 6 years to answer the following questions: (1) How well do these models forecast the arrival time of CME-driven shocks? (2) What are the uncertainties associated with these forecasts? (3) Which model(s) perform best? (4) Have the models become more accurate during the past 6 years? We analyze all forecasts made by 32 models from 2013 through mid-2018, and additionally focus on 28 events, all of which were forecasted by six models. We find that the models are generally able to predict CME-shock arrival times, in an average sense, to within ±10 hr, but with standard deviations often exceeding 20 hr. The best performers, on the other hand, maintained a mean error (bias) of 1 hr, a mean absolute error of 13 hr, and a precision (standard deviation) of 15 hr. Finally, there is no evidence that the forecasts have become more accurate during this interval. We discuss the intrinsic simplifications of thevarious models analyzed, the limitations of this investigation, and suggest possible paths to improve these forecasts in the future.

space weather↗

Learning turbulent flows with generative models for super resolution and sparse flow reconstruction

Neural operators are promising surrogates for dynamical systems but when trained with standard L 2 losses they tend to oversmooth fine-scale turbulent structures. Here, we show that combining operator learning with generative modeling overcomes this limitation. We consider three practical turbulent-flow challenges where conventional neural operators fail: spatio-temporal super-resolution, forecasting, and sparse flow reconstruction. For Schlieren jet super-resolution, an adversarially trained neural operator (adv-NO) reduces the energy-spectrum error by 15 × while preserving sharp gradients at neural operator-like inference cost. For 3D homogeneous isotropic turbulence, adv-NO trained on only 160 timesteps from a single trajectory forecasts accurately for five eddy-turnover times and offers 114 × wall-clock speed-up at inference than the baseline diffusion-based forecasters, enabling near-real-time rollouts. For reconstructing cylinder wake flows from highly sparse Particle Tracking Velocimetry-like inputs, a conditional generative model infers full 3D velocity and pressure fields with correct phase alignment and statistics. These advances enable accurate reconstruction and forecasting at low compute cost, bringing near-real-time analysis and control within reach in experimental and computational fluid mechanics.

Fluid dynamics↗

Differing Air Traffic Controller Responses to Similar Trajectory Prediction Errors

A Human-In-The-Loop simulation was conducted in January of 2013 in the Airspace Operations Laboratory at NASA's Ames Research Center. The simulation airspace included two en route sectors feeding the northwest corner of Atlanta's Terminal Radar Approach Control. The focus of this paper is on how uncertainties in the study's trajectory predictions impacted the controllers ability to perform their duties. Of particular interest is how the controllers interacted with the delay information displayed in the meter list and data block while managing the arrival flows. Due to wind forecasts with 30-knot over-predictions and 30-knot under-predictions, delay value computations included errors of similar magnitude, albeit in opposite directions. However, when performing their duties in the presence of these errors, did the controllers issue clearances of similar magnitude, albeit in opposite directions?

trajectory prediction uncertainty↗

Automating Rabi & Ramsey Measurements via ML

As quantum computers scale up, the manual process of qubit tune-up becomes increasingly impractical due to its time-consuming and repetitive nature. While existing research has explored some automation techniques, many models remain underutilized for this purpose. This research aims to answer the question: is qubit tune-up able to be automated using the Long Short-Term Memory (LSTM) model? For the purposes of this project, only the rabi and ramsey measurement cycle was automated. These measurements are used to fine-tune a rough qubit frequency by repeating them until the optimal qubit frequency is obtained. The LSTM model uses the qubit frequency at one time step to forecast the qubit frequency at the next time step. A rabi-ramsey simulation was made to fabricate a dataset to train and test the LSTM model. As the model was trained, the error of the model decreased. Although there wasn't enough training data to generate perfect predictions, this shows it is possible to utilize forecasting models in automating the tune-up process.

Roberts, Rachel↗

Automating Rabi & Ramsey Measurements via ML

As quantum computers scale up, the manual process of qubit tune-up becomes increasingly impractical due to its time-consuming and repetitive nature. While existing research has explored some automation techniques, many models remain underutilized for this purpose. This research aims to answer the question: can qubit tune-up be automated using the Long Short-Term Memory (LSTM) model? For the purposes of this project, only the rabi and ramsey measurement cycle was automated. These measurements are used to fine-tune a rough qubit frequency by repeating them until the optimal qubit frequency is obtained. The LSTM model uses the qubit frequency at one time step to forecast the qubit frequency at the next time step. A rabi-ramsey simulation was made to fabricate a dataset to train and test the LSTM model. As the model was trained, the error of the model decreased. Although there wasn t enough training data to generate perfect predictions, this shows it is possible to utilize forecasting models in automating the tune-up process.

Roberts, Rachel↗

Automating Rabi & Ramsey Measurements via Machine Learning

As quantum computers scale up, the manual process of qubit tune-up becomes increasingly impractical due to its time-consuming and repetitive nature. While existing research has explored some automation techniques, many models remain underutilized for this purpose. This research aims to answer the question: can qubit tune-up be automated using the Long Short-Term Memory (LSTM) model? For the purposes of this project, only the rabi and ramsey measurement cycle was automated. These measurements are used to fine-tune a rough qubit frequency by repeating them until the optimal qubit frequency is obtained. The LSTM model uses the qubit frequency at one time step to forecast the qubit frequency at the next time step. A rabi-ramsey simulation was made to fabricate a dataset to train and test the LSTM model. As the model was trained, the error of the model decreased. Although there wasn't enough training data to generate perfect predictions, this shows it is possible to utilize forecasting models in automating the tune-up process.

Roberts, Rachel↗

Differing Air Traffic Controller Responses to Similar Trajectory Prediction Errors: An Interrupted Time-Series Analysis of Controller Behavior

A Human-In-The-Loop simulation was conducted in January of 2013 in the Airspace Operations Laboratory at NASAs Ames Research Center. The simulation airspace included two en route sectors feeding the northwest corner of Atlantas Terminal Radar Approach Control. The focus of this paper is on how uncertainties in the studys trajectory predictions impacted the controllers ability to perform their duties. Of particular interest is how the controllers interacted with the delay information displayed in the meter list and data block while managing the arrival flows. Due to wind forecasts with 20-knot over-predictions and 20-knot under-predictions, delay value computations included errors of similar magnitude, albeit in opposite directions. However, when performing their duties in the presence of these errors, did the controllers issue clearances of similar magnitude, albeit in opposite directions?

human-automation interaction↗

Modeling of Multiscale Solar Dynamics for Understanding Drivers of Space Weather

Understanding the solar dynamics is critical for improving our capabilities to forecast the evolution of space weather conditions. We take advantage of currently available computational capabilities to model solar dynamics from the deep interior to the corona and investigate mechanisms that may drive space weather conditions. The simulations are performed using the 3D radiative MHD code StellarBox. Comparison of synthetic spectroscopic observables obtained from numerical simulations and actual observations allows us to uncover physical processes associated with observed phenomena. To facilitate a transition from modeling short-term physical phenomena to developing a reliable forecast-oriented model, we suggest using the data assimilation approach. It allows us to cross-analyze dynamo model solutions and observations and to consider possible uncertainties and errors. In this presentation, we briefly summarize current multi-scale modeling capabilities and results and discuss ongoing developments to build a reliable physics-based forecast-oriented model of solar activity.

Irina N Kitiashvili↗

Modeling of Multiscale Solar Dynamics for Understanding Drivers of Space Weather

Understanding the solar dynamics is critical for improving our capabilities to forecast the evolution of space weather conditions. We take advantage of currently available computational capabilities to model solar dynamics from the deep interior to the corona and investigate mechanisms that may drive space weather conditions. The simulations are performed using the 3D radiative MHD code StellarBox. Comparison of synthetic spectroscopic observables obtained from numerical simulations and actual observations allows us to uncover physical processes associated with observed phenomena. To facilitate a transition from modeling short-term physical phenomena to developing a reliable forecast-oriented model, we suggest using the data assimilation approach. It allows us to cross-analyze dynamo model solutions and observations and to consider possible uncertainties and errors. In this presentation, we briefly summarize current multi-scale modeling capabilities and results and discuss ongoing developments to build a reliable physics-based forecast-oriented model of solar activity

Irina N Kitiashvili↗

Modeling of Multiscale Solar Dynamics for Understanding Drivers of Space Weather

Understanding the solar dynamics is critical for improving our capabilities to forecast the evolution of space weather conditions. We take advantage of currently available computational capabilities to model solar dynamics from the deep interior to the corona and investigate mechanisms that may drive space weather conditions. The simulations are performed using the 3D radiative MHD code StellarBox. Comparison of synthetic spectroscopic observables obtained from numerical simulations and actual observations allows us to uncover physical processes associated with observed phenomena. To facilitate a transition from modeling short-term physical phenomena to developing a reliable forecast-oriented model, we suggest using the data assimilation approach. It allows us to cross-analyze dynamo model solutions and observations and to consider possible uncertainties and errors. In this presentation, we briefly summarize current multi-scale modeling capabilities and results and discuss ongoing developments to build a reliable physics-based forecast-oriented model of solar activity

Irina N. Kitiashvili↗

The Impact of Ocean Data Assimilation on Seasonal-to-Interannual Forecasts: A Case Study of the 2006 El Nino Event

This study investigates the impact of four different ocean analyses on coupled forecasts of the 2006 El Nino event. Forecasts initialized in June 2006 using ocean analyses from an assimilation that uses flow-dependent background error covariances are compared with those using static error covariances that are not flow dependent. The flow-dependent error covariances reflect the error structures related to the background ENSO instability and are generated by the coupled breeding method. The ocean analyses used in this study result from the assimilation of temperature and salinity, with the salinity data available from Argo floats. Of the analyses, the one using information from the coupled bred vectors (BV) replicates the observed equatorial long wave propagation best and exhibits more warming features leading to the 2006 El Nino event. The forecasts initialized from the BV-based analysis agree best with the observations in terms of the growth of the warm anomaly through two warming phases. This better performance is related to the impact of the salinity analysis on the state evolution in the equatorial thermocline. The early warming is traced back to salinity differences in the upper ocean of the equatorial central Pacific, while the second warming, corresponding to the mature phase, is associated with the effect of the salinity assimilation on the depth of the thermocline in the western equatorial Pacific. The series of forecast experiments conducted here show that the structure of the salinity in the initial conditions is important to the forecasts of the extension of the warm pool and the evolution of the 2006 El Ni o event.

Yang, Shu-Chih↗

Short-Term Solar Forecasting Performance of Popular Machine Learning Algorithms: Preprint

A framework for assessing the performance of short-term solar forecasting is presented in conjunction with a range of numerical results using global horizontal irradiation (GHI) from the open-source Surface Radiation Budget (SURFRAD) data network. A suite of popular machine learning algorithms is compared according to a set of statistically distinct metrics and benchmarked against the persistence-of-cloudiness forecast and a cloud motion forecast. Results show significant improvement compared to the benchmarks with trade-offs among the machine learning algorithms depending on the desired error metric. Training inputs include time series observations of GHI for a history of years, historical weather and atmospheric measurements, and corresponding date and time stamps such that training sensitivities might be inferred. Prediction outputs are GHI forecasts for 1, 2, 3, and 4 hours ahead of the issue time, and they are made for every month of the year for 7 locations. Photovoltaic power and energy outputs can then be made using the solar forecasts to better understand power system impacts.

14 SOLAR ENERGY↗

Some Issues in Uncertainty Quantification and Parameter Tuning: A Case Study of Convective Parameterization Scheme in the WRF Regional Climate Model

The current tuning process of parameters in global climate models is often performed subjectively, or treated as an optimization procedure to minimize the difference between model fields and observations. The later approach may be generating a set of tunable parameters that approximate the observed climate but via an unrealistic balance of physical processes and/or compensating errors over different regions in the globe. In this study, we run the Weather Research and Forecasting (WRF) regional model constrained by the reanalysis data over the Southern Great Plains (SGP) where abundant observational data from various resources are available for calibration of the input parameters and validation of the model results. Our goal is to quantify the uncertainty ranges and identify the optimal values of five key input parameters in a new Kain-Frisch (KF) convective parameterization scheme incorporated in the WRF model. A stochastic sampling algorithm, Multiple Very Fast Simulated Annealing (MVFSA), is employed to efficiently sample the input parameters in KF scheme based on the skill score so that the algorithm progressively moves toward regions of the parameter space that minimize model errors. The results based on the WRF simulations with 25-km grid spacing over the SGP show that the model bias for precipitation can be significantly reduced by using five optimal parameters identified by the MVFSA algorithm. The model performance is very sensitive to downdraft and entrainment related parameters and consumption time of Convective Available Potential Energy (CAPE). Simulated convective precipitation decreases as the ratio of downdraft to updraft flux increases. Larger CAPE consumption time results in less convective but more stratiform precipitation. The simulation using optimal parameters obtained by only constraining precipitation generates positive impact on the other output variables, such as temperature and wind. By using the optimal parameters obtained at 25 km simulation, both the magnitude and spatial pattern of simulated precipitation are improved at 12-km spatial resolution. The optimal parameters identified from the SGP region have also improved the simulation of precipitation when moving model domain to another region with a different climate regime (i.e. North America monsoon region). These results suggest the improvement of precipitation simulation by using the optimal parameters remains when the model domain or spatial resolution is changed.

54 ENVIRONMENTAL SCIENCES↗