Search NASASearch

SEARCH · Search NASA

Results for “ensemble learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

A novel conditional generative model for efficient ensemble forecasts of state variables in large-scale geological carbon storage

Integrating monitoring data to efficiently update reservoir pressure and CO 2 plume distribution forecasts presents a significant challenge in geological carbon storage (GCS) applications. Inverse modeling techniques are commonly used to fuse observational data and refine reservoir model parameters, thereby improving state variable forecasts. However, these techniques often rely on linear or Gaussian assumptions, which can limit their effectiveness in accurately predicting state variables. Moreover, simulating large-scale three-dimensional (3D) GCS problems is computationally expensive, making iterative runs in inverse problems prohibitive. To address these challenges, we propose a conditional generative model utilizing the score-based diffusion method for real-time 3D pressure and saturation field distribution predictions. Our approach involves solving the score function with a mini-batch-based Monte Carlo estimator to generate labeled data. This data is subsequently employed to train a fully connected neural network, enabling it to learn the conditional sample generator within a supervised learning framework. This method enables the rapid generation of a large ensemble of predictions, facilitating comprehensive uncertainty quantification of state variables. Here we applied our method to forecast the dynamic 3D distributions of pressure and saturation fields over a 30-year injection period. The statistical assessment with low root mean square error (RMSE) values demonstrates that our method can accurately predict the spatiotemporal distributions of both pressure and saturation fields. Moreover, the developed conditional generative model shows high computational efficiency by generating 100 ensemble forecasts of 3D state variables in less than 10 min. The consistency between ensemble averages and ground truth values further illustrates the model’s capability to capture state variable dynamics during the CO 2 plume injection process. Notably, the ground truth values fall within the ensemble forecasts, indicating that our uncertainty quantification effectively captures variability and potential noise in the observations. Thus, the developed conditional generative model proves to be a more efficient, accurate, and practical tool for GCS applications, facilitating timely risk analysis and informed decision-making.

58 GEOSCIENCES

Nanoscale wetting controls reactive Pd ensembles in synthesis of dilute PdAu alloy catalysts

The performance of bimetallic dilute alloy catalysts is largely determined by the size of minority metal ensembles on the nanoparticle surface. By analyzing the synthesis of catalysts comprising Pd 8 Au 92 nanoparticles supported on silica using surface-sensitive techniques, we report that whether Pd overgrowth occurs before or after Au nanoparticle deposition onto the support controls the surface Pd ensemble size and abundance. These differences in Pd ensembles influence catalytic reactivity in H 2 –D 2 isotope exchange and benzaldehyde hydrogenation, which, in correlation with theoretical calculations, is used to elucidate the active site(s) in each reaction. To clarify how the synthetic sequence controls the formation of Pd ensembles, we combine numerical wetting calculations and molecular dynamics simulations (with a machine-learned force field) to visualize Pd deposition and migration on the nanoparticle surface, respectively. Our results suggest that the nanoparticle–support interface restricts nanoparticle accessibility to Pd deposition, which consequently controls the Pd ensemble size, illustrating the critical role of nanoscale wetting phenomena during bimetallic catalyst preparation.

36 MATERIALS SCIENCE

Scalable learning of potentials to predict time-dependent Hartree–Fock dynamics

We propose a framework to learn the time-dependent Hartree–Fock (TDHF) inter-electronic potential of a molecule from its electron density dynamics. Although the entire TDHF Hamiltonian, including the inter-electronic potential, can be computed from first principles, we use this problem as a testbed to develop strategies that can be applied to learn a priori unknown terms that arise in other methods/approaches to quantum dynamics, e.g., emerging problems such as learning exchange–correlation potentials for time-dependent density functional theory. We develop, train, and test three models of the TDHF inter-electronic potential, each parameterized by a four-index tensor of size up to 60 × 60 × 60 × 60. Two of the models preserve Hermitian symmetry, while one model preserves an eight-fold permutation symmetry that implies Hermitian symmetry. Across seven different molecular systems, we find that accounting for the deeper eight-fold symmetry leads to the best-performing model across three metrics: training efficiency, test set predictive power, and direct comparison of true and learned inter-electronic potentials. All three models, when trained on ensembles of field-free trajectories, generate accurate electron dynamics predictions even in a field-on regime that lies outside the training set. To enable our models to scale to large molecular systems, we derive expressions for Jacobian-vector products that enable iterative, matrix-free training.

97 MATHEMATICS AND COMPUTING

Integrating State Data Assimilation and Innovative Model Parameterization Reduces Simulated Carbon Uptake in the Arctic and Boreal Region

Model representation of carbon uptake and storage is essential for accurate projection of the response of the arctic-boreal zone to a rapidly changing climate. Land model estimates of LAI and aboveground biomass that can have a marked influence on model projections of carbon uptake and storage vary substantially in the arctic and boreal zone, making it challenging to correctly evaluate model estimates of Gross Primary Productivity (GPP). To understand and correct bias of LAI and aboveground biomass in the Community Land Model (CLM), we assimilated the 8-day Moderate Resolution Imaging Spectroradiometer (MODIS) LAI observation and a machine learning product of annual aboveground biomass into CLM using an Ensemble Adjustment Kalman Filter (EAKF) in an experimental region including Alaska and Western Canada. Assimilating LAI and aboveground biomass reduced these model estimates by 58% and 72%, respectively. The change of aboveground biomass was consistent with independent estimates of canopy top height at both regional and site levels. The International Land Model Benchmarking system assessment showed that data assimilation significantly improved CLM's performance in simulating the carbon and hydrological cycles, as well as in representing the functional relationships between LAI and other variables. Here, to further reduce the remaining bias in GPP after LAI bias correction, we re-parameterized CLM to account for low temperature suppression of photosynthesis. The LAI bias corrected model that included the new parameterization showed the best agreement with model benchmarks. Combining data assimilation with model parameterization provides a useful framework to assess photosynthetic processes in LSMs.

58 GEOSCIENCES

Integrating State Data Assimilation and Innovative Model Parameterization Reduces Simulated Carbon Uptake in the Arctic and Boreal Region

Model representation of carbon uptake and storage is essential for accurate projection of the response of the arctic‐boreal zone to a rapidly changing climate. Land model estimates of LAI and aboveground biomass that can have a marked influence on model projections of carbon uptake and storage vary substantially in the arctic and boreal zone, making it challenging to correctly evaluate model estimates of Gross Primary Productivity (GPP). To understand and correct bias of LAI and aboveground biomass in the Community Land Model (CLM), we assimilated the 8‐day Moderate Resolution Imaging Spectroradiometer (MODIS) LAI observation and a machine learning product of annual aboveground biomass into CLM using an Ensemble Adjustment Kalman Filter (EAKF) in an experimental region including Alaska and Western Canada. Assimilating LAI and aboveground biomass reduced these model estimates by 58% and 72%, respectively. The change of aboveground biomass was consistent with independent estimates of canopy top height at both regional and site levels. The International Land Model Benchmarking system assessment showed that data assimilation significantly improved CLM's performance in simulating the carbon and hydrological cycles, as well as in representing the functional relationships between LAI and other variables. To further reduce the remaining bias in GPP after LAI bias correction, we re‐parameterized CLM to account for low temperature suppression of photosynthesis. The LAI bias corrected model that included the new parameterization showed the best agreement with model benchmarks. Combining data assimilation with model parameterization provides a useful framework to assess photosynthetic processes in LSMs.

land data assimilation

Quantifying uncertainty in machine learning for nuclear binding energy

Techniques from artificial intelligence and machine learning are increasingly employed in nuclear theory; however, the uncertainties that arise from the complex parameter manifold encoded by the neural networks are often overlooked. Epistemic uncertainties arising from training the same network multiple times for an ensemble of initial weight sets offer a first insight into the confidence of machine learning predictions, but they often come with a high computational cost. Instead, we apply a single-model uncertainty quantification method called Δ-UQ that gives epistemic uncertainties with one-time training. Here, we demonstrate our approach on a two-feature model of nuclear binding energies per nucleon with proton and neutron number pairs as inputs. We show that Δ-UQ can produce reliable and self-consistent epistemic uncertainty estimates and can be used to assess the degree of confidence in predictions made with deep neural networks.

Huang, Mengyao [Lawrence Livermore National Labora

On learning what to learn: Heterogeneous observations of dynamics and establishing possibly causal relations among them

Abstract Before we attempt to (approximately) learn a function between two sets of observables of a physical process, we must first decide what the inputs and outputs of the desired function are going to be. Here we demonstrate two distinct, data-driven ways of first deciding “the right quantities” to relate through such a function, and then proceeding to learn it. This is accomplished by first processing simultaneous heterogeneous data streams (ensembles of time series) from observations of a physical system: records of multiple observation processes of the system. We determine (i) what subsets of observables are common between the observation processes (and therefore observable from each other, relatable through a function); and (ii) what information is unrelated to these common observables, therefore particular to each observation process, and not contributing to the desired function. Any data-driven technique can subsequently be used to learn the input–output relation—from k-nearest neighbors and Geometric Harmonics to Gaussian Processes and Neural Networks. Two particular “twists” of the approach are discussed. The first has to do with the identifiability of particular quantities of interest from the measurements. We now construct mappings from a single set of observations from one process to entire level sets of measurements of the second process, consistent with this single set. The second attempts to relate our framework to a form of causality: if one of the observation processes measures “now,” while the second observation process measures “in the future,” the function to be learned among what is common across observation processes constitutes a dynamical model for the system evolution.

Sroczynski, David W.

Using Ensemble Decisions and Active Selection to Improve Low-Cost Labeling for Multi-View Data

This paper seeks to improve low-cost labeling in terms of training set reliability (the fraction of correctly labeled training items) and test set performance for multi-view learning methods. Co-training is a popular multiview learning method that combines high-confidence example selection with low-cost (self) labeling. However, co-training with certain base learning algorithms significantly reduces training set reliability, causing an associated drop in prediction accuracy. We propose the use of ensemble labeling to improve reliability in such cases. We also discuss and show promising results on combining low-cost ensemble labeling with active (low-confidence) example selection. We unify these example selection and labeling strategies under collaborative learning, a family of techniques for multi-view learning that we are developing for distributed, sensor-network environments.

machine learning

Quantifying Groundwater Response and Uncertainty in Beaver‐Influenced Mountainous Floodplains Using Machine Learning‐Based Model Calibration

Abstract Beavers ( Castor canadensis ) alter river corridor hydrology by creating ponds and inundating floodplains, and thereby improving surface water storage. However, the impact of inundation on groundwater, particularly in mountainous alluvial floodplains with permeable gravel/cobble layers overlain by a soil layer, remains uncertain. Numerical modeling across various floodplain structures considers topographic and sediment complexity and multidirectional flow, linking inundation to groundwater response. This study develops a model‐data integration workflow to address uncertainty in groundwater response to beaver‐induced inundations in a mountainous alluvial floodplain in the Upper Colorado River Basin. Uncertain factors include seasonal hydrologic dynamics, hydraulic conductivities, floodplain structures, and meteorological forcings. We employed an ensemble of groundwater models, based on geophysical and hydrologic data, with machine learning‐based calibration using a neural density estimator. This allowed us to quantify the vertical flux from the soil layer to the permeable gravel bed, the down‐valley underflow within the gravel bed, and their ratios. Results show a significant increase in the vertical flux relative to down‐valley underflow, from 2 during dry pond periods to 20 during wet periods, serving as an analogy for conditions without and with beaver ponds. The study highlights the influence of floodplain structure on groundwater storage, water balance, and water quality impacted by beaver ponds. A thick gravel bed layer, with a large down‐valley underflow, minimizes the effect of beaver‐induced inundation on water quality. We emphasize the need for field‐scale measurements of floodplain structure and improved characterization of evapotranspiration changes to reduce uncertainty in groundwater response. Plain Language Summary Beavers change the flow of water in river corridors by creating ponds, expanding wetlands, and flooding floodplains. This increases surface water area, promotes plant growth, and enhances biodiversity. However, the impact of this flooding on groundwater flow is not well understood, especially in mountainous areas with gravel layers where water moves easily beneath soil. In this study, we used numerical modeling to investigate how beaver ponds influence groundwater in a mountainous floodplain of the Upper Colorado River Basin. We adapted a machine learning method to validate our numerical models using multiple field data sets. Our findings show that beaver ponds significantly increase vertical water flow from the soil to the gravel during wet periods, compared to when the ponds are fully drained. The study also highlights the importance of floodplain structure in controlling both water flow in gravel layers along the river direction and vertical flow from the soil to the gravel with the presence of beavers. To reduce uncertainty in groundwater response, we emphasize the need for more field‐scale measurements of floodplain structure, hydraulic properties, and evapotranspiration changes. Key Points Floodplain structures and hydraulic conductivities are important for groundwater response with beaver ponds in mountainous floodplains Large down‐valley underflow in permeability‐stratified floodplains reduces beaver‐induced impacts on groundwater storage and water quality Machine learning‐based model calibration methods are effective for estimating posterior distributions of groundwater model parameters

Wang, Lijing

Portable, heterogeneous ensemble workflows at scale using libEnsemble

libEnsemble is a Python-based toolkit for running dynamic ensembles, developed as part of the DOE Exascale Computing Project. The toolkit utilizes a unique generator–simulator–allocator paradigm, where generators produce input for simulators, simulators evaluate those inputs, and allocators decide whether and when a simulator or generator should be called. The generator steers the ensemble based on simulation results. Generators may, for example, apply methods for numerical optimization, machine learning, or statistical calibration. libEnsemble communicates between a manager and workers. Flexibility is provided through multiple manager–worker communication substrates each of which has different benefits. These include Python’s multiprocessing, mpi4py, and TCP. Multisite ensembles are supported using Balsam or Globus Compute. We overview the unique characteristics of libEnsemble as well as current and potential interoperability with other packages in the workflow ecosystem. We highlight libEnsemble’s dynamic resource features: libEnsemble can detect system resources, such as available nodes, cores, and GPUs, and assign these in a portable way. These features allow users to specify the number of processors and GPUs required for each simulation; and resources will be automatically assigned on a wide range of systems, including Frontier, Aurora, and Perlmutter. Such ensembles can include multiple simulation types, some using GPUs and others using only CPUs, sharing nodes for maximum efficiency. We also describe the benefits of libEnsemble’s generator–simulator coupling, which easily exposes to the user the ability to cancel, and portably kill, running simulations based on models that are updated with intermediate simulation output. We demonstrate libEnsemble’s capabilities, scalability, and scientific impact via a Gaussian process surrogate training problem for the longitudinal density profile at the exit of a plasma accelerator stage. In conclusion, the study uses gpCAM for the surrogate model and employs either Wake-T or WarpX simulations, highlighting efficient use of resources that can easily extend to exascale.

Dynamic ensembles

Unbinned extraction of $γ$ from $B\to DK$ with normalizing flows

We introduce an unbinned method for extracting the CKM angle $γ$ from the decay chain $B^\pm \to (D \to K_S π^+ π^-) K^\pm$ using normalizing flows (NFs). The NFs, trained on $D$ decay data, learn a faithful continuous representation of the amplitude and strong phase variation over the $D\to K_Sπ^+π^-$ Dalitz plot whose fidelity improves with increased data sample sizes. With this input, the $B$ decay data can be used to extract the parameters $r_B$, $δ_B$, and $γ$. We test the method on Monte Carlo generated data, where it successfully recovers the injected value of $γ$ within uncertainties. The present implementation propagates statistical uncertainties from finite training data via an ensemble of independently trained flows, and does not attempt to capture the effects of systematic experimental errors. We explore two versions of the method that differ in how the trigonometric constraint on phase variation is encoded, and comment on the possible extension to Bayesian NFs, which would provide direct uncertainty estimates on the learned densities without requiring ensemble training.

Grossman, Yuval [Cornell U., LEPP]

Pushing the frontiers in climate modelling and analysis with machine learning

Climate modelling and analysis are facing new demands to enhance projections and climate information. Here, in this study, we argue that now is the time to push the frontiers of machine learning beyond state-of-the-art approaches, not only by developing machine-learning-based Earth system models with greater fidelity, but also by providing new capabilities through emulators for extreme event projections with large ensembles, enhanced detection and attribution methods for extreme events, and advanced climate model analysis and benchmarking. Utilizing this potential requires key machine learning challenges to be addressed, in particular generalization, uncertainty quantification, explainable artificial intelligence and causality. This interdisciplinary effort requires bringing together machine learning and climate scientists, while also leveraging the private sector, to accelerate progress towards actionable climate science.

54 ENVIRONMENTAL SCIENCES

Automated Knowledge Discovery From Simulators

A computational method, SimLearn, has been devised to facilitate efficient knowledge discovery from simulators. Simulators are complex computer programs used in science and engineering to model diverse phenomena such as fluid flow, gravitational interactions, coupled mechanical systems, and nuclear, chemical, and biological processes. SimLearn uses active-learning techniques to efficiently address the "landscape characterization problem." In particular, SimLearn tries to determine which regions in "input space" lead to a given output from the simulator, where "input space" refers to an abstraction of all the variables going into the simulator, e.g., initial conditions, parameters, and interaction equations. Landscape characterization can be viewed as an attempt to invert the forward mapping of the simulator and recover the inputs that produce a particular output. Given that a single simulation run can take days or weeks to complete even on a large computing cluster, SimLearn attempts to reduce costs by reducing the number of simulations needed to effect discoveries. Unlike conventional data-mining methods that are applied to static predefined datasets, SimLearn involves an iterative process in which a most informative dataset is constructed dynamically by using the simulator as an oracle. On each iteration, the algorithm models the knowledge it has gained through previous simulation trials and then chooses which simulation trials to run next. Running these trials through the simulator produces new data in the form of input-output pairs. The overall process is embodied in an algorithm that combines support vector machines (SVMs) with active learning. SVMs use learning from examples (the examples are the input-output pairs generated by running the simulator) and a principle called maximum margin to derive predictors that generalize well to new inputs. In SimLearn, the SVM plays the role of modeling the knowledge that has been gained through previous simulation trials. Active learning is used to determine which new input points would be most informative if their output were known. The selected input points are run through the simulator to generate new information that can be used to refine the SVM. The process is then repeated. SimLearn carefully balances exploration (semi-randomly searching around the input space) versus exploitation (using the current state of knowledge to conduct a tightly focused search). During each iteration, SimLearn uses not one, but an ensemble of SVMs. Each SVM in the ensemble is characterized by different hyper-parameters that control various aspects of the learned predictor - for example, whether the predictor is constrained to be very smooth (nearby points in input space lead to similar output predictions) or whether the predictor is allowed to be "bumpy." The various SVMs will have different preferences about which input points they would like to run through the simulator next. SimLearn includes a formal mechanism for balancing the ensemble SVM preferences so that a single choice can be made for the next set of trials.

Burl, Michael

Dimensionality Reduction Through Classifier Ensembles

In data mining, one often needs to analyze datasets with a very large number of attributes. Performing machine learning directly on such data sets is often impractical because of extensive run times, excessive complexity of the fitted model (often leading to overfitting), and the well-known "curse of dimensionality." In practice, to avoid such problems, feature selection and/or extraction are often used to reduce data dimensionality prior to the learning step. However, existing feature selection/extraction algorithms either evaluate features by their effectiveness across the entire data set or simply disregard class information altogether (e.g., principal component analysis). Furthermore, feature extraction algorithms such as principal components analysis create new features that are often meaningless to human users. In this article, we present input decimation, a method that provides "feature subsets" that are selected for their ability to discriminate among the classes. These features are subsequently used in ensembles of classifiers, yielding results superior to single classifiers, ensembles that use the full set of features, and ensembles based on principal component analysis on both real and synthetic datasets.

Oza, Nikunj C.

Conformalized-KANs: Uncertainty Quantification with Coverage Guarantees for Kolmogorov-Arnold Networks (KANs) in Scientific Machine Learning

This paper explores uncertainty quantification (UQ) methods in the context of Kolmogorov–Arnold Networks (KANs). We apply an ensemble approach to KANs to obtain a heuristic measure of UQ, enhancing interpretability and robustness in modeling complex functions. Building on this, we introduce Conformalized-KANs, which integrate conformal prediction, a distribution-free UQ technique, with KAN ensembles to generate calibrated prediction intervals with guaranteed coverage.} Extensive numerical experiments are conducted to evaluate the effectiveness of these methods, focusing particularly on the robustness and accuracy of the prediction intervals under various hyperparameter settings. We show that the conformal KAN predictions can be applied to recent extensions of KANs, including Finite Basis KANs (FBKANs) and multifideilty KANs (MFKANs). The results demonstrate the potential of our approaches to significantly improve the reliability and applicability of KANs in scientific machine learning.

• Artificial intelligence (AI) / machine learning

Graph reinforcement learning for exploring model spaces beyond the standard model

We present a methodology for performing scans of beyond the standard model (BSM) parameter spaces with reinforcement learning. We identify a novel procedure using graph neural networks that is capable of exploring spaces of models without the user specifying a fixed particle content, allowing broad classes of BSM models to be explored—in theory, the technique is applicable to nearly any model space with a prespecified gauge group. We provide a generic procedure by which a suitable graph grammar can be developed for any BSM model that features user-specified symmetry groups and a finite number of different possible particle species, the use of which is applicable to a variety of machine learning tasks over the actions of BSM theories beyond our particular reinforcement learning use case. As a proof of concept, we construct the graph grammar for theories with vectorlike leptons that may or may not be charged under a dark U ( 1 ) group, inspired by portal matter extensions of the sub-GeV vector portal/kinetic mixing simplified dark matter models. We then use this graph grammar to create a reinforcement learning environment tasked with creating models with these vectorlike leptons that are consistent with a list of a variety of precision observables. The reinforcement learning agent succeeds in developing models that can address the observed muon anomalous magnetic moment discrepancy while remaining consistent with flavor violation and electroweak precision observables, including both constructions that have previously been studied as well as new models that have not, to our knowledge, previously been identified. By inspecting the resulting ensembles of models that the agent produces and experimenting with different configurations for our reinforcement learning environment and graph grammar, we also infer various lessons about the development of these environments that can be transferable to reinforcement learning scans of more complicated model spaces and comment on future directions for the development of this technique into a more mature tool. Published by the American Physical Society 2025

Wojcik, George N.

Continual learning in the presence of repetition

Continual learning (CL) provides a framework for training models in ever-evolving environments. Although re-occurrence of previously seen objects or tasks is common in real-world problems, the concept of repetition in the data stream is not often considered in standard benchmarks for CL. Unlike with the rehearsal mechanism in buffer-based strategies, where sample repetition is controlled by the strategy, repetition in the data stream naturally stems from the environment. This report provides a summary of the CLVision challenge at CVPR 2023, which focused on the topic of repetition in class-incremental learning. The report initially outlines the challenge objective and then describes three solutions proposed by finalist teams that aim to effectively exploit the repetition in the stream to learn continually. The experimental results from the challenge highlight the effectiveness of ensemble-based solutions that employ multiple versions of similar modules, each trained on different but overlapping subsets of classes. This report underscores the transformative potential of taking a different perspective in CL by employing repetition in the data stream to foster innovative strategy design.

Class-incremental learning

Transfer learning of neural surrogates on multifidelity groundwater simulations

Multifidelity data used in the paper published in Advances in Water Resources 206 (2025) 105140, https://doi.org/10.1016/j.advwatres.2025.105140 The code used to process the data is openly available on GitHub at https://github.com/Model-Reduction-and-UQ-Group/Transfer_Learning_K_reconstruction Computationally inexpensive surrogates of process-based models, such as deep neural networks, enable ensemble-based computations used in risk assessment, data assimilation, etc. However, generation of large datasets required to train a neural network can be as expensive as the ensemble simulations themselves. We ameliorate this challenge by using data from multifidelity (MF) groundwater simulations and transfer learning (TL) to reduce data generation costs while maintaining model accuracy. As a computational example, we train a deep convolutional neural network (CNN) to reconstruct permeability fields from saturation maps derived from a multiphase flow model. Starting with very low- and low-fidelity data generated on increasingly coarse meshes, we pretrain the CNN, followed by output-layer training and fine-tuning using only a limited number of high-fidelity samples. We demonstrate the surrogate’s robustness when interpreting low-quality inputs—such as interpolated maps or data affected by noise—which has strong implications for the applicability in practical hydrogeological scenarios. This multilevel MF-TL strategy achieves a favorable trade-off between computational efficiency and predictive accuracy, significantly outperforming high-fidelity-only approaches under the same computational budget.

Chiofalo, Alessia [University of Bologna] (ORCID:0