Beyond Leaderboards: Tokenomics of Agentic Small Language Model Ensembles
Explore the source record for details and available documents.
97 MATHEMATICS AND COMPUTING↗
SEARCH · Search NASA
Results for “Ensemble model”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Advancing research on compound weather and climate events via large ensemble model simulations
Societally relevant weather impacts typically result from compound events, which are rare combinations of weather and climate drivers. Focussing on four event types arising from different combinations of climate variables across space and time, here we illustrate that robust analyses of compound events — such as frequency and uncertainty analysis under present-day and future conditions, event attribution to climate change, and exploration of low-probability-high-impact events — require data with very large sample size. In particular, the required sample is much larger than that needed for analyses of univariate extremes. We demonstrate that Single Model Initial-condition Large Ensemble (SMILE) simulations from multiple climate models, which provide hundreds to thousands of years of weather conditions, are crucial for advancing our assessments of compound events and constructing robust model projections. Combining SMILEs with an improved physical understanding of compound events will ultimately provide practitioners and stakeholders with the best available information on climate risks.
54 ENVIRONMENTAL SCIENCES↗
Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations
Background: Short-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here, we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022. Methods: We used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported by a standardised source for 32 countries over the next 1–4 weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models’ predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models’ forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models’ past predictive performance. Results: Over 52 weeks, we collected forecasts from 48 unique models. We evaluated 29 models’ forecast scores in comparison to the ensemble model. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 83% of participating models’ forecasts of incident cases (with a total N=886 predictions from 23 unique models), and 91% of participating models’ forecasts of deaths (N=763 predictions from 20 models). Across a 1–4 week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over 4 weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models. Conclusions: Our results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than 2 weeks. Funding: AA, BH, BL, LWa, MMa, PP, SV funded by National Institutes of Health (NIH) Grant 1R01GM109718, NSF BIG DATA Grant IIS-1633028, NSF Grant No.: OAC-1916805, NSF Expeditions in Computing Grant CCF-1918656, CCF-1917819, NSF RAPID CNS-2028004, NSF RAPID OAC-2027541, US Centers for Disease Control and Prevention 75D30119C05935, a grant from Google, University of Virginia Strategic Investment Fund award number SIF160, Defense Threat Reduction Agency (DTRA) under Contract No. HDTRA1-19-D-0007, and respectively Virginia Dept of Health Grant VDH-21-501-0141, VDH-21-501-0143, VDH-21-501-0147, VDH-21-501-0145, VDH-21-501-0146, VDH-21-501-0142, VDH-21-501-0148. AF, AMa, GL funded by SMIGE - Modelli statistici inferenziali per governare l'epidemia, FISR 2020-Covid-19 I Fase, FISR2020IP-00156, Codice Progetto: PRJ-0695. AM, BK, FD, FR, JK, JN, JZ, KN, MG, MR, MS, RB funded by Ministry of Science and Higher Education of Poland with grant 28/WFSN/2021 to the University of Warsaw. BRe, CPe, JLAz funded by Ministerio de Sanidad/ISCIII. BT, PG funded by PERISCOPE European H2020 project, contract number 101016233. CP, DL, EA, MC, SA funded by European Commission - Directorate-General for Communications Networks, Content and Technology through the contract LC-01485746, and Ministerio de Ciencia, Innovacion y Universidades and FEDER, with the project PGC2018-095456-B-I00. DE., MGu funded by Spanish Ministry of Health / REACT-UE (FEDER). DO, GF, IMi, LC funded by Laboratory Directed Research and Development program of Los Alamos National Laboratory (LANL) under project number 20200700ER. DS, ELR, GG, NGR, NW, YW funded by National Institutes of General Medical Sciences (R35GM119582; the content is solely the responsibility of the authors and does not necessarily represent the official views of NIGMS or the National Institutes of Health). FB, FP funded by InPresa, Lombardy Region, Italy. HG, KS funded by European Centre for Disease Prevention and Control. IV funded by Agencia de Qualitat i Avaluacio Sanitaries de Catalunya (AQuAS) through contract 2021-021OE. JDe, SMo, VP funded by Netzwerk Universitatsmedizin (NUM) project egePan (01KX2021). JPB, SH, TH funded by Federal Ministry of Education and Research (BMBF; grant 05M18SIA). KH, MSc, YKh funded by Project SaxoCOV, funded by the German Free State of Saxony. Presentation of data, model results and simulations also funded by the NFDI4Health Task Force COVID-19 ( https://www.nfdi4health.de/task-force-covid-19-2 ) within the framework of a DFG-project (LO-342/17-1). LP, VE funded by Mathematical and Statistical modelling project (MUNI/A/1615/2020), Online platform for real-time monitoring, analysis and management of epidemic situations (MUNI/11/02202001/2020); VE also supported by RECETOX research infrastructure (Ministry of Education, Youth and Sports of the Czech Republic: LM2018121), the CETOCOEN EXCELLENCE (CZ.02.1.01/0.0/0.0/17-043/0009632), RECETOX RI project (CZ.02.1.01/0.0/0.0/16-013/0001761). NIB funded by Health Protection Research Unit (grant code NIHR200908). SAb, SF funded by Wellcome Trust (210758/Z/18/Z).
60 APPLIED LIFE SCIENCES↗
Pyrocumulonimbus Events Over British Columbia in 2017: An Ensemble Model Study of Parameter Sensitivities and Climate Impacts of Wildfire Smoke in the Stratosphere
Abstract The Pyrocumulonimbus (pyroCb) events over British Columbia in 2017 were observed in the lower stratosphere for about 8–10 months after the smoke injections. Several previous studies used global climate models to investigate the physical parameters for the 2017 pyroCb events, but the conclusions show strong model dependency. In this study, we use the Energy Exascale Earth System Model (E3SM) and complete an ensemble of runs exploring three injection parameters: smoke aerosol mass, the percentage of black carbon within the smoke aerosols, and plume injection height. Additionally, we consider the heterogeneous reaction of ozone and primary organic matter. According to the satellite daily observed aerosol optical depth (AOD), we find that the best ensemble member is the simulation with 0.4 Tg of smoke, 3% of which is black carbon, a 13.5 km smoke injection height, and a 10 −5 probability factor of the heterogeneous reaction. Besides AOD, we examine the ensemble score based on the metrics used in previous studies: the metrics of the extinction coefficient at 18 km altitude and the maximum plume height. The conclusion of the best estimate of the injection parameters for 2017 pyroCb events shows strong not only model but also evaluation metric dependency. We use the Random Forest machine learning technique to quantify the relative importance of each parameter in accurately simulating the 2017 pyroCb events and find that the injection height is the most critical feature, no matter which metric is used to score the ensemble members.
54 ENVIRONMENTAL SCIENCES↗
Prioritizing the selection of CMIP6 model ensemble members for downscaling projections of CONUS temperature and precipitation
Given the mismatch between the large volume of data archived for the sixth phase of the Coupled Model Intercomparison Project (CMIP6) and limited personnel and computational resources for downscaling, only a small fraction of the CMIP6 archive can be downscaled. In this work, we develop an approach to robustly sample projected hydroclimate states in CMIP6 for downscaling to test whether the selection of a single initial condition (IC) ensemble member from each CMIP6 model is sufficient to span the range of modeled hydroclimate over the conterminous United States (CONUS) and CONUS sub-regions. We calculate the pattern-centered root mean square difference of IC ensemble member anomalies relative to each model’s historical climatology for shared socioeconomic pathway (SSP) projections over 30-year time periods and compare the ratio of inter-model to intra-model variability for this metric. Regardless of SSP, inter-model variability is generally much greater than intra-model variability at the scales of the CONUS as a whole, as well as for most CONUS sub-regions. However, for some variables and scenarios, inter- and intra-model variability are similar at sub-CONUS scales, indicating that selecting a single IC ensemble member per model may be sufficient to sample the range of projected hydroclimate states in the 21st Century across CONUS, but for specific regions and variables, more careful selection of ensemble members may be necessary. Regionally resolved Taylor diagrams identify where more IC ensemble member downscaling efforts should be focused if resources are available to do so. Our results suggest that, with parsimonious sampling, the requisite computational expense of downscaling temperature and precipitation fields over the CONUS for subsequent CMIP activities may increase only marginally despite the great increase in data volumes with each successive CMIP phase.
54 ENVIRONMENTAL SCIENCES↗
Uncertainty quantification of mass models using ensemble Bayesian model averaging
Developments in the description of the masses of atomic nuclei have led to various nuclear mass models that provide predictions for masses across the whole chart of nuclides. These mass models play an important role in understanding the synthesis of heavy elements in the rapid neutron capture ( r ) process. However, it is still a challenging task to estimate the size of uncertainty associated with the predictions of each mass model. In this work, a method called ensemble Bayesian model averaging (EBMA) is introduced to quantify the uncertainty of one-neutron separation energies (S 1 n ) which are directly relevant in the calculations of r -process observables. Here, this Bayesian method provides a natural way to perform model averaging, selection, and uncertainty quantification, by combining the mass models as a mixture of normal distributions whose parameters are optimized against the experimental data, employing the Markov chain Monte Carlo method using the no-u-turn sampler. The EBMA model optimized with all the experimental S 1 n from the AME2003 nuclides are shown to provide reliable uncertainty estimates when tested with the new data in the AME2020.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
A framework for ensemble modelling of climate change impacts on lakes worldwide: the ISIMIP Lake Sector
Abstract. Empirical evidence demonstrates that lakes and reservoirs are warming across the globe. Consequently, there is an increased need to project future changes in lake thermal structure and resulting changes in lake biogeochemistry in order to plan for the likely impacts. Previous studies of the impacts of climate change on lakes have often relied on a single model forced with limited scenario-driven projections of future climate for a relatively small number of lakes. As a result, our understanding of the effects of climate change on lakes is fragmentary, based on scattered studies using different data sources and modelling protocols, and mainly focused on individual lakes or lake regions. This has precluded identification of the main impacts of climate change on lakes at global and regional scales and has likely contributed to the lack of lake water quality considerations in policy-relevant documents, such as the Assessment Reports of the Intergovernmental Panel on Climate Change (IPCC). Here, we describe a simulation protocol developed by the Lake Sector of the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP) for simulating climate change impacts on lakes using an ensemble of lake models and climate change scenarios for ISIMIP phases 2 and 3. The protocol prescribes lake simulations driven by climate forcing from gridded observations and different Earth system models under various representative greenhouse gas concentration pathways (RCPs), all consistently bias-corrected on a 0.5∘ × 0.5∘ global grid. In ISIMIP phase 2, 11 lake models were forced with these data to project the thermal structure of 62 well-studied lakes where data were available for calibration under historical conditions, and using uncalibrated models for 17 500 lakes defined for all global grid cells containing lakes. In ISIMIP phase 3, this approach was expanded to consider more lakes, more models, and more processes. The ISIMIP Lake Sector is the largest international effort to project future water temperature, thermal structure, and ice phenology of lakes at local and global scales and paves the way for future simulations of the impacts of climate change on water quality and biogeochemistry in lakes.
58 GEOSCIENCES↗
Historical and future trends in South Asian monsoon low pressure systems in a high-resolution model ensemble
Historical trends in monsoon low pressure systems (LPS), the dominant rain-bearing weather system of South Asia, have been difficult to assess due to changes in the observing network. Future projections have also remained uncertain because prior studies concluded that many coarse-resolution climate models do not accurately simulate LPS. Here, we examine changes in South Asian monsoon LPS simulated by an ensemble of global models, including some with high spatial resolution, that we show skillfully represent LPS. In the ensemble mean, the number of strong LPS (monsoon depressions) decreased over the last 65 years (1950–2014) by about 15% while no trend was detected for weaker LPS (monsoon lows). The reduction in depression counts then moderated, yielding no trend in the periods 1980–2050 or 2015–2050. The ensemble mean projects a shift in genesis from ocean to land and an increase in LPS precipitation of at least 7% K −1 , which together contribute to a projected increase in seasonal mean and extreme precipitation over central India.
54 ENVIRONMENTAL SCIENCES↗
ISMIP6 Antarctica: a multi-model ensemble of the Antarctic ice sheet evolution over the 21st century
Abstract. Ice flow models of the Antarctic ice sheet are commonly used to simulate its future evolution in response to different climate scenarios and assess the mass loss that would contribute to future sea level rise. However, there is currently no consensus on estimates of the future mass balance of the ice sheet, primarily because of differences in the representation of physical processes, forcings employed and initial states of ice sheet models. This study presents results from ice flow model simulations from 13 international groups focusing on the evolution of the Antarctic ice sheet during the period 2015–2100 as part of the Ice Sheet Model Intercomparison for CMIP6 (ISMIP6). They are forced with outputs from a subset of models from the Coupled Model Intercomparison Project Phase 5 (CMIP5), representative of the spread in climate model results. Simulations of the Antarctic ice sheet contribution to sea level rise in response to increased warming during this period varies between −7.8 and 30.0 cm of sea level equivalent (SLE) under Representative Concentration Pathway (RCP) 8.5 scenario forcing. These numbers are relative to a control experiment with constant climate conditions and should therefore be added to the mass loss contribution under climate conditions similar to present-day conditions over the same period. The simulated evolution of the West Antarctic ice sheet varies widely among models, with an overall mass loss, up to 18.0 cm SLE, in response to changes in oceanic conditions. East Antarctica mass change varies between −6.1 and 8.3 cm SLE in the simulations, with a significant increase in surface mass balance outweighing the increased ice discharge under most RCP 8.5 scenario forcings. The inclusion of ice shelf collapse, here assumed to be caused by large amounts of liquid water ponding at the surface of ice shelves, yields an additional simulated mass loss of 28 mm compared to simulations without ice shelf collapse. The largest sources of uncertainty come from the climate forcing, the ocean-induced melt rates, the calibration of these melt rates based on oceanic conditions taken outside of ice shelf cavities and the ice sheet dynamic response to these oceanic changes. Results under RCP 2.6 scenario based on two CMIP5 climate models show an additional mass loss of 0 and 3 cm of SLE on average compared to simulations done under present-day conditions for the two CMIP5 forcings used and display limited mass gain in East Antarctica.
54 ENVIRONMENTAL SCIENCES↗
A Machine Learning Framework for Modeling Ensemble Properties of Atomically Disordered Materials
Atomic disorder can strongly influence material properties such as charge transport, optical response, and catalytic activity. However, efficiently modeling these disorder effects remains challenging for first-principles methods due to the cost of sampling large configurational spaces and computing complex physical quantities. Recent advances of machine learning techniques, particularly graph neural networks (GNNs), has enabled the efficient and accurate predictions of complex material properties, offering promising tools for studying disordered systems. In this work, we present a general machine-learning-assisted computational framework that integrates equivariant GNNs with Monte Carlo simulations to compute the thermodynamic and ensemble-averaged functional properties of disordered materials. Using the surface-termination-disordered MXene monolayer Ti 3 C 2 T 2–x as a representative system, we find that electrical conductivity exhibits an emergent peak near the order–disorder phase transition temperature due to the interplay between electron scattering and doping. In contrast, optical conductivity remains largely insensitive to local atomic disorder and reflects the global surface chemical composition. These results highlight the role of atomic disorder in affecting material properties and demonstrate the potential of our approach for statistically modeling disorder effects in a wide range of materials such as high-entropy alloys and spin liquids.
MXene↗
Inferring the Redshift of More than 150 GRBs with a Machine-learning Ensemble Model
Gamma-ray bursts (GRBs), due to their high luminosities, are detected up to a redshift of 10, and thus have the potential to be vital cosmological probes of early processes in the Universe. Fulfilling this potential requires a large sample of GRBs with known redshifts, but due to observational limitations, only 11% have known redshifts (z). There have been numerous attempts to estimate redshifts via correlation studies, most of which have led to inaccurate predictions. To overcome this, we estimated GRB redshift via an ensemble-supervised machine-learning (ML) model that uses X-ray afterglows of long-duration GRBs observed by the Neil Gehrels Swift Observatory. The estimated redshifts are strongly correlated (a Pearson coefficient of 0.93) and have an rms error, namely, the square root of the average squared error $\langle$Δz 2 $\rangle$, of 0.46 with the observed redshifts showing the reliability of this method. The addition of GRB afterglow parameters improves the predictions considerably by 63% compared to previous results in peer-reviewed literature. Finally, we use our ML model to infer the redshifts of 154 GRBs, which increase the known redshifts of long GRBs with plateaus by 94%, a significant milestone for enhancing GRB population studies that require large samples with redshift.
79 ASTRONOMY AND ASTROPHYSICS↗
Identifying precursors of daily to seasonal hydrological extremes over the USA using deep learning techniques and climate model ensembles
Focal Area(s): We focus on two areas of crosscutting interest for DOE: 1) predictability of extreme precipitation and drought in the USA and 2) the integration of climate models with new AI tools, such as convolutional neural networks (CNN) and methods to understand their output (e.g. layer-wise relevance propagation; LRP). This project fits into focus area 3 of this call for white paper using AI to gain insight from complex data, including explainable AI tools. Science Challenge: Predicting hydrological extremes is important due to their impacts on people, agriculture and infrastructure. This prediction is difficult due to the infrequent occurrence of extremes and their complexity. However, extreme events can be related to more predictable conditions in the ocean, such as El Nino, long-term soil moisture or large scale modes of climate variability, such as the North Atlantic Oscillation (NAO).
54 ENVIRONMENTAL SCIENCES↗
Insights into the vulnerability of Antarctic glaciers from the ISMIP6 ice sheet model ensemble and associated uncertainty
Abstract. The Antarctic Ice Sheet represents the largest source of uncertainty in future sea level rise projections, with a contribution to sea level by 2100 ranging from −5 to 43 cm of sea level equivalent under high carbon emission scenarios estimated by the recent Ice Sheet Model Intercomparison for CMIP6 (ISMIP6). ISMIP6 highlighted the different behaviors of the East and West Antarctic ice sheets, as well as the possible role of increased surface mass balance in offsetting the dynamic ice loss in response to changing oceanic conditions in ice shelf cavities. However, the detailed contribution of individual glaciers, as well as the partitioning of uncertainty associated with this ensemble, have not yet been investigated. Here, we analyze the ISMIP6 results for high carbon emission scenarios, focusing on key glaciers around the Antarctic Ice Sheet, and we quantify their projected dynamic mass loss, defined here as mass loss through increased ice discharge into the ocean in response to changing oceanic conditions. We highlight glaciers contributing the most to sea level rise, as well as their vulnerability to changes in oceanic conditions. We then investigate the different sources of uncertainty and their relative role in projections, for the entire continent and for key individual glaciers. We show that, in addition to Thwaites and Pine Island glaciers in West Antarctica, Totten and Moscow University glaciers in East Antarctica present comparable future dynamic mass loss and high sensitivity to ice shelf basal melt. The overall uncertainty in additional dynamic mass loss in response to changing oceanic conditions, compared to a scenario with constant oceanic conditions, is dominated by the choice of ice sheet model, accounting for 52 % of the total uncertainty of the Antarctic dynamic mass loss in 2100. Its relative role for the most dynamic glaciers varies between 14 % for MacAyeal and Whillans ice streams and 56 % for Pine Island Glacier at the end of the century. The uncertainty associated with the choice of climate model increases over time and reaches 13 % of the uncertainty by 2100 for the Antarctic Ice Sheet but varies between 4 % for Thwaites Glacier and 53 % for Whillans Ice Stream. The uncertainty associated with the ice–climate interaction, which captures different treatments of oceanic forcings such as the choice of melt parameterization, its calibration, and simulated ice shelf geometries, accounts for 22 % of the uncertainty at the ice sheet scale but reaches 36 % and 39 % for Institute Ice Stream and Thwaites Glacier, respectively, by 2100. Overall, this study helps inform future research by highlighting the sectors of the ice sheet most vulnerable to oceanic warming over the 21st century and by quantifying the main sources of uncertainty.
54 ENVIRONMENTAL SCIENCES↗
A Model Ensemble Approach Enables Data-Driven Property Prediction for Chemically Deconstructable Thermosets in the Low-Data Regime
Not Available
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Ensemble models of feedstock blend ratios to minimize supply chain risk in bio-based manufacturing
Not Available
Chen, Chyi-Shin↗
Ensemble modeling of global lake evaporation under climate change
Global projections of lake evaporation are typically based on simulations using single mechanistic models. However, because of its complex interactions with various lake physical properties, environmental and anthropogenic drivers, lake evaporation is highly variable and sensitive to the choice of model used. In this study, we present a multi-model analysis to investigate differences across global simulations of lake evaporation during the warm-season using three different lake models driven by outputs from four general circulation models (GCM) (i.e. 12 model combinations in total) for historic and future scenarios. Our results suggest substantial differences among lake-climate model simulations of lake evaporation. These differences varied throughout the 20th and 21st century, with model driver data explaining 74% of the variance in future projections of warm- season lake evaporation. Our projections indicate that, by the end of the 21st century (2070–2099), global annual lake evaporation rates will increase by 10–27% under Representative Concentration Pathways (RCPs) 2.6–8.5. Finally, we highlight the importance of using a multi-model approach for the prediction of future global lake evaporation responses to climate change.
54 ENVIRONMENTAL SCIENCES↗
Ensemble models for circuit topology estimation, fault detection and classification in distribution systems
This paper presents a methodology for simultaneous fault detection, classification, and topology estimation for adaptive protection of distribution systems. The methodology estimates the probability of the occurrence of each one of these events by using a hybrid structure that combines three sub-systems, a convolutional neural network for topology estimation, a fault detection based on predictive residual analysis, and a standard support vector machine with probabilistic output for fault classification. The input to all these sub-systems is the local voltage and current measurements. A convolutional neural network uses these local measurements in the form of sequential data to extract features and estimate the topology conditions. The fault detector is constructed with a Bayesian stage (a multitask Gaussian process) that computes a predictive distribution (assumed to be Gaussian) of the residuals using the input. Since the distribution is known, these residuals can be transformed into a Standard distribution, whose values are then introduced into a one-class support vector machine. The structure allows using a one-class support vector machine without parameter cross-validation, so the fault detector is fully unsupervised. Finally, a support vector machine uses the input to perform the classification of the fault types. All three sub-systems can work in a parallel setup for both performance and computation efficiency. In conclusion, we test all three sub-systems included in the structure on a modified IEEE123 bus system, and we compare and evaluate the results with standard approaches.
24 POWER TRANSMISSION AND DISTRIBUTION↗