Search NASA⌕ Search

SEARCH · Search NASA

Results for “interval score”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

12 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Evaluation of the 2022 West Nile virus forecasting challenge, USA

Abstract Background West Nile virus (WNV) is the most common cause of mosquito-borne disease in the continental USA, with an average of ~1200 severe, neuroinvasive cases reported annually from 2005 to 2021 (range 386–2873). Despite this burden, efforts to forecast WNV disease to inform public health measures to reduce disease incidence have had limited success. Here, we analyze forecasts submitted to the 2022 WNV Forecasting Challenge, a follow-up to the 2020 WNV Forecasting Challenge. Methods Forecasting teams submitted probabilistic forecasts of annual West Nile virus neuroinvasive disease (WNND) cases for each county in the continental USA for the 2022 WNV season. We assessed the skill of team-specific forecasts, baseline forecasts, and an ensemble created from team-specific forecasts. We then characterized the impact of model characteristics and county-specific contextual factors (e.g., population) on forecast skill. Results Ensemble forecasts for 2022 anticipated a season at or below median long-term WNND incidence for nearly all (> 99%) counties. More counties reported higher case numbers than anticipated by the ensemble forecast median, but national caseload (826) was well below the 10-year median (1386). Forecast skill was highest for the ensemble forecast, though the historical negative binomial baseline model and several team-submitted forecasts had similar forecast skill. Forecasts utilizing regression-based frameworks tended to have more skill than those that did not and models using climate, mosquito surveillance, demographic, or avian data had less skill than those that did not, potentially due to overfitting. County-contextual analysis showed strong relationships with the number of years that WNND had been reported and permutation entropy (historical variability). Evaluations based on weighted interval score and logarithmic scoring metrics produced similar results. Conclusions The relative success of the ensemble forecast, the best forecast for 2022, suggests potential gains in community ability to forecast WNV, an improvement from the 2020 Challenge. Similar to the previous challenge, however, our results indicate that skill was still limited with general underprediction despite a relative low incidence year. Potential opportunities for improvement include refining mechanistic approaches, integrating additional data sources, and considering different approaches for areas with and without previous cases. Graphical Abstract

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Evaluation of FluSight influenza forecasting in the 2021–22 and 2022–23 seasons with a new target laboratory-confirmed influenza hospitalizations

Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. For the 2021–22 and 2022–23 influenza seasons, 26 forecasting teams provided national and jurisdiction-specific probabilistic predictions of weekly confirmed influenza hospital admissions for one-to-four weeks ahead. Forecast skill is evaluated using the Weighted Interval Score (WIS), relative WIS, and coverage. Six out of 23 models outperform the baseline model across forecast weeks and locations in 2021–22 and 12 out of 18 models in 2022–23. Averaging across all forecast targets, the FluSight ensemble is the 2nd most accurate model measured by WIS in 2021–22 and the 5th most accurate in the 2022–23 season. Forecast skill and 95% coverage for the FluSight ensemble and most component models degrade over longer forecast horizons. In this work we demonstrate that while the FluSight ensemble was a robust predictor, even ensembles face challenges during periods of rapid change.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Synthetic method of analogues for emerging infectious disease forecasting

The Method of Analogues (MOA) has gained popularity in the past decade for infectious disease forecasting due to its non-parametric nature. In MOA, the local behavior observed in a time series is matched to the local behaviors of several historical time series. The known values that directly follow the historical time series that best match the observed time series are used to calculate a forecast. This non-parametric approach leverages historical trends to produce forecasts without extensive parameterization, making it highly adaptable. However, MOA is limited in scenarios where historical data is sparse. This limitation was particularly evident during the early stages of the COVID-19 pandemic, where the emerging global epidemic had little-to-no historical data. In this work, we propose a new method inspired by MOA, called the Synthetic Method of Analogues (sMOA). sMOA replaces historical disease data with a library of synthetic data that describe a broad range of possible disease trends. This model circumvents the need to estimate explicit parameter values by instead matching segments of ongoing time series data to a comprehensive library of synthetically generated segments of time series data. We demonstrate that sMOA has competitive performance with state-of-the-art infectious disease forecasting models, out-performing 78% of models from the COVID-19 Forecasting Hub in terms of averaged Mean Absolute Error and 76% of models from the COVID-19 Forecasting Hub in terms of averaged Weighted Interval Score. Additionally, we introduce a novel uncertainty quantification methodology designed for the onset of emerging epidemics. Developing versatile approaches that do not rely on historical data and can maintain high accuracy in the face of novel pandemics is critical for enhancing public health decision-making and strengthening preparedness for future outbreaks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

The Sensitivity of Variational Bayesian Neural Network Performance to Hyperparameters

In scientific applications, predictive modeling is often of limited use without accurate uncertainty quantification (UQ) to indicate when a model may be extrapolating or when more data needs to be collected. Bayesian Neural Networks (BNNs) produce predictive uncertainty by propagating uncertainty in neural network (NN) weights and offer the promise of obtaining not only an accurate predictive model but also accurate UQ. However, in practice, obtaining accurate UQ with BNNs is difficult due in part to the approximations used for model training (such as those made in variational inference) and in part to the need to choose a suitable set of hyperparameters; these hyperparameters outnumber those needed for traditional NNs and often have opaque effects on the results. We aim to shed light on the effects of hyperparameter choices for variational BNNs by performing a global sensitivity analysis of variational BNN performance under varying hyperparameter settings. Our results indicate that many of the hyperparameters interact with each other to affect both predictive accuracy and UQ. For improved usage of variational BNNs in real-world applications, we suggest that thorough hyperparameter tuning, including tuning of prior hyperparameters and loss function parameters, is essential for accurate UQ in variational BNNs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Uncertainty in Synthetic Tropical Cyclone Hazard and Risk Estimates: Insights from RAFT, CHAZ, MIT, STORM, and CLIMADA

We synthesize five complementary tropical cyclone (TC) hazard frameworks—RAFT (physics-based machine learning), CHAZ and MIT (statistical–dynamical), STORM (fully statistical), and CLIMADA (observation-driven resampling)—to characterize uncertainty in wind-related TC metrics relevant to energy applications. All datasets and the IBTrACS observational record are harmonized to a common 6-hourly, 2.5° grid. We compare basin-wide and coastal properties using consistent definitions for TC frequency, mean and maximum intensity, 24-hour intensification, and 6-hour translation speed, and quantify agreement with Pearson r, RMSE, and Kling–Gupta efficiency (KGE) alongside resampling-based confidence intervals. CLIMADA is included for basin context but excluded from coastal skill scoring because it resamples historical IBTrACS; if supplied with projected future tracks from an external hazard model, CLIMADA can be used to simulate future TC scenarios. Results show robust, cross-model signals: (i) a corridor of activity from the tropical Atlantic through the Caribbean into the Bahamas and western subtropical Atlantic; (ii) a meridional dipole in 24-hour intensification (low-latitude strengthening, subtropical weakening); and (iii) a transition from slower tropical motion to faster midlatitude translation. Coastal winds (mean and maximum) consistently cluster from the eastern Gulf into the Bahamas–western Atlantic transition. The largest structural spread occurs in the amplitude and footprint of lifetime maximum intensity and, secondarily, in translation speed; intensification exhibits similar central behavior across frameworks with variability in extremes. Translation speed shows the most uniform coastal agreement. These findings provide a decision envelope for wind-focused risk screening and clarify where uncertainty should be carried forward; wind-only results represent a lower bound on total hazard, motivating integration of surge and rainfall modules and a companion, asset-level damage analysis.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Early calendar life and health prediction of silicon batteries via machine learning with uncertainty quantification

Lithium-ion batteries with silicon anodes promise high energy density but are limited by calendar lifetime. Reducing the long iteration time to obtain experimental results requires predicting calendar lifetime early in a cell's life. In this study, we demonstrate that lightweight machine learning models with feature engineering can provide calendar lifetime estimates from early electrochemical signals. After 1 month of electrochemical aging, the best models achieve 10% error in calendar-life prediction and can separate "bad" from "good" lifetime cells with a mean F1 score of 0.857. As battery systems exhibit inherent variability, four methods for uncertainty quantification are compared, and confidence intervals are demonstrated with an uncertainty of +-3.6 months in lifetime prediction. A feature importance analysis indicates that early patterns in voltage decay are the strongest indicators of calendar lifetime. Finally, this modeling approach has high error when generalizing to new electrode chemistries or testing conditions but with appropriately low confidence.

25 ENERGY STORAGE↗
DOE OSTI2025

Uncertainty quantification of a physics-informed model based on sparse identification of a Thermal Energy Distribution System

Integrated energy systems (IES)s are crucial for enhancing the economy and efficiency of power generation sources (e.g., nuclear energy) necessary to unleash American energy dominance. These systems can be integrated with thermal energy storage (TES) and intermittent renewable energies to optimize overall energy use, peak-load regulation, and demand-side responses. However, the stabilization of energy generation, transport, and utilization introduces operational complexities that exceed the challenges of managing each sub-component individually. Currently, though IESs rely on human operators for efficiency and stability, reducing human error risk and enhancing performance through automation is highly desirable. Recent advances at Idaho National Laboratory have demonstrated successful control of the Thermal Energy Distributed System (TEDS). However, the automatic control system depends on a deterministic Sparse Identification of Nonlinear Dynamics with Control (SINDyC) model, which are trained based on simulation data from physics-based simulations. Because of uncertainties in physics-based simulation, SINDyC model results in large discrepancies against experimental data and cannot be reliably used in automatic control. In this paper, we present an innovative approach to address these discrepancies by quantifying uncertainties and developing a more robust model. We first generated trajectories by using first-principles physics codes to encapsulate the experiment. Next, we trained thousands of models by randomly sampling these trajectories. We then collapsed all those models into one probabilistic SINDyC by fitting a multivariate Gaussian distribution onto the resulting coefficient’s distribution. Despite its simplicity, our approach successfully produced 95% confidence intervals that captured the experimental trajectories. It even did so with a higher probability and better U-pooling score across six of the seven relevant quantities of interest (QoIs), as compared to other classical approaches. In conclusion, ongoing research is focusing on generating new experimental trajectories to validate this approach, and on employing Bayesian calibration to refine parametric uncertainties and guide future model development efforts.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Fine-Grained Power and Energy Attribution on AMD GPU/APU-Based Exascale Nodes

Modern exascale GPU- and APU-based systems provide multiple power and energy sensors, but differences in scope, update rate, timing, and filtering complicate the attribution of short-lived accelerator activity. This paper presents a methodology to characterize and correct these effects on Cray EX systems with AMD Instinct MI250X GPUs (Frontier) and MI300A APUs (Portage). Using controlled square-wave workloads, we quantify update intervals, delay, aliasing, and variability across up to 512 GPUs and 480 APUs with on-chip (rocm-smi/amd-smi) and off-chip Cray Power Management sensors. We reconstruct power from cumulative energy counters to achieve faster response times, validate it against on-chip, off-chip, and node-level sensors, and integrate the resulting streams into a Score-P/PAPI-based tool for time-aligned, phase-level attribution. Applied to rocHPL, rocHPL-MxP, and HPG-MxP, the method separates energy savings due to reduced runtime from changes in power. Mixed precision reduces node energy on Frontier by 79% for rocHPL-MxP and 31% for HPG-MxP, with similar trends on Portage. These results provide portable guidance for sensor validation and power-aware optimization on current and future exascale systems.

Mcdaniel, Adam [ORNL] (ORCID:000000016926028X)↗
DOE OSTI2024

Machine Learning-Driven Quantification of CO2 Plume Dynamics at Illinois Basin Decatur Project Sites Using Microseismic Data

This study utilizes machine learning to quantify CO2 plume extents by analyzing microseismic data from the Illinois Basin Decatur Project (IBDP). Leveraging a unique dataset of well logs, microseismic records, and CO2 injection metrics, this work aims to predict the temporal evolution of subsurface CO2 saturation plumes. The findings illustrate that machine learning can predict plume dynamics, revealing vertical clustering of microseismic events over distinct time periods within certain proximities to the injection well, consistent with an invasion percolation model. The buoyant CO2 plume partially trapped within sandstone intervals periodically breaches localized barriers or baffles, which act as leaky seals and impede vertical migration until buoyancy overcomes gravity and capillary forces, leading to breakthroughs along vertical zones of weakness. Between different unsupervised clustering techniques, K-Means and DBSCAN were applied and analyzed in detail, where K-means outperformed DBSCAN in this specific study by indicating the combination of the highest Silhouette Score and the lowest Davies–Bouldin Index. The predictive capability of machine learning models in quantifying CO2 saturation plume extension is significant for real-time monitoring and management of CO2 sequestration sites. The models exhibit high accuracy, validated against physical models and injection data from the IBDP, reinforcing the viability of CO2 geological sequestration as a climate change mitigation strategy and enhancing advanced tools for safe management of these operations.

Iyegbekedo, Ikponmwosa↗
DOE OSTI2025

Transcripts and genomic intervals associated with variation in metabolite abundance in maize leaves under field conditions

Abstract Plants exhibit extensive environment-dependent intraspecific metabolic variation, which likely plays a role in determining variation in whole plant phenotypes. However, much of the work seeking to use natural variation to link genes and transcript’s impacts on plant metabolism has employed data from controlled environments. Here, we generated and analyzed data on the variation in the abundance of 26 metabolites across 660 maize inbred lines under field conditions. We employ these data and previously published transcript and whole plant phenotype data reported for the same field experiment to identify both genomic intervals (through genome-wide association studies (GWAS)) and transcripts (using both transcriptome-wide association studies (TWAS) and an explainable artificial intelligence (AI) approach based on random forest (RF)) associated with variation in metabolite abundance. Both genome-wide association and random forest-based methods identified substantial numbers of significant associations including genes with plausible links to the metabolites they are associated with. In contrast, the transcriptome-wide association identified only six significant associations. In three cases, genetic markers associated with metabolic variation in our study colocalized with markers linked to variation in non-metabolic traits scored in the same experiment. We speculate that the poor performance of transcriptome-wide association studies in identifying transcript-metabolite associations may reflect a high prevalence of non-linear interactions between transcripts and metabolites and/or a bias towards rare transcripts playing a large role in determining intraspecific metabolic variation.

Mathivanan, Ramesh Kanna↗
DOE OSTI2025

Detection of Isotopes in Urban Source Search Low-Count Gamma Spectra Using Hopfield Neural Networks

Source search campaigns involve measurements of background gamma-ray spectra with a mobile detector-spectrometer traveling along arbitrarily chosen trajectories over a wide screening area. Radiation counts are typically measured with a tellurium-doped sodium iodide [NaI(Tl)] scintillator detector-spectrometer in short acquisition intervals, usually 1 s. The objective is to detect orphan isotopes with half-lives shorter than those of the isotopes in the natural background. In principle, radioisotopes can be identified by their unique gamma emission spectrum. However, detecting orphan isotopes in search data is challenging because low counts measured in short acquisition intervals result in incomplete spectral lines. In this study, we investigate the performance of a Hopfield neural network (HNN) that implements an auto-associative memory for the detection of isotopes of interest in an urban search campaign. The HNN is trained on one example of gamma spectra with well-resolved spectral lines of each isotope of interest. During testing, the auto-associative memory implementation of the HNN processes low-count gamma spectra with partially complete isotopic lines by matching incoming measurements to the closest one of its memory-stored patterns. The testing database consisted of almost 10 000 1-s gamma spectra, including measurements of orphan isotopes 137 Cs, 241 Am, and 131 I, obtained during two urban search surveys with a NaI(Tl) detector. The performance of the HNN detection algorithm was evaluated using precision, recall, and F1 scores, and benchmarked with a multiple linear regression (MLR) identification algorithm. In conclusion, the test results demonstrate that HNN outperforms MLR in the detection of all the isotopes of interest.

Auto associative memory↗
DOE OSTI2026

A Machine Learning based Approach of Estimating Equivalent Circuit Model Parameters at Different SoCs of Li-ion Batteries from Voltage Relaxation

Abstract: In this study, an approach of estimating the equivalent circuit model (ECM) parameters for Li-ion batteries (LIBs) is proposed based on the voltage value at different intervals while relaxing the LIB after discharge. The typical approach for estimating ECM parameters of a LIB is to conduct electrochemical impedance spectroscopy (EIS) measurements at different frequencies and fit them to a predefined circuit model, which requires additional measuring arrangements and specialized devices. The proposed methodology utilizes four different voltages at 0s, 60s, 360s, and 1800s alongside the specific state of charge (SoC) value for a specific constant discharge current value of ~1C until the relaxation stage to train and evaluate three regression-based machine learning models— Support Vector Regression (SVR), Extreme Gradient Boosting (XGBoost), and Gaussian Process Regression (GPR)—for estimating the ECM parameters of the selected model. Bayesian optimization is employed for hyperparameter tuning to achieve optimal performance for all the regressor models, among which, the GPR provided the best performance with the root-mean-squared error (RMSE) of less than 4x10-4 on average for the resistive components and less than 0.27 for capacitive components with excellent R2 scores. The simplicity of the approach enables it to eliminate the need for sophisticated measuring equipment and computation power.

Sagar, Md. Samiul [The University of Alabama (UA)]↗