Search NASA⌕ Search

SEARCH · Search NASA

Results for “Ensemble computing.”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Applications of Ensemble-based Data Assimilation Techniques for Aquifer Characterization using Tracer Data at Hanford 300 Area

Subsurface aquifer characterization often involves high parameter dimensionality and requires tremendous computational resources if employing a full Bayesian approach. Ensemble-based data assimilation techniques, including filtering and smoothing, are computationally efficient alternatives. Despite the increasing number of applications of ensemble-based methods in assimilating flow and transport related data for subsurface aquifer charaterization, most are limited to either synthetic studies or two-dimensional problems. In this study, we applied ensemble-based techniques for assimilating field tracer experimental data obtained from the Integrated Field Research Challenge (IFRC) site at the Hanford 300 Area. The forward problem was simulated using the massively-parallel three-dimensional flow and transport code PFLOTRAN to effectively deal with the highly transient flow boundary conditions at the site and to meet the computational demands of ensemble-based methods. This study demonstrates the effectiveness of ensemble-based methods for characterizing a heterogeneous aquifer by sequentially assimilating multiple types of data. The necessity of employing high performance computing is shown to enable increasingly mechanistic non-linear forward simulations to be performed within the data assimilation framework for a complex system with reasonable turnaround time.

data assimilation, ensemble kalman filter, inverse↗

Hierarchical ensemble Kalman methods with sparsity-promoting generalized gamma hyperpriors

This paper introduces a computational framework to incorporate flexible regularization techniques in ensemble Kalman methods, generalizing the iterative alternating scheme to nonlinear inverse problems. The proposed methodology approximates the maximum a posteriori (MAP) estimate of a hierarchical Bayesian model characterized by a conditionally Gaussian prior and generalized gamma hyperpriors. Suitable choices of hyperparameters yield sparsity-promoting regularization. We propose an iterative algorithm for MAP estimation, which alternates between updating the unknown with an ensemble Kalman method and updating the hyperparameters in the regularization to promote sparsity. Here, the effectiveness of our methodology is demonstrated in several computed examples, including compressed sensing and subsurface flow inverse problems.

Ensemble Kalman methods↗

Reduced-order autodifferentiable ensemble Kalman filters

This paper introduces a computational framework to reconstruct and forecast a partially observed state that evolves according to an unknown or expensive-to-simulate dynamical system. Our reduced-order autodifferentiable ensemble Kalman filters (ROAD-EnKFs) learn a latent low-dimensional surrogate model for the dynamics and a decoder that maps from the latent space to the state space. The learned dynamics and decoder are then used within an EnKF to reconstruct and forecast the state. Numerical experiments show that if the state dynamics exhibit a hidden low-dimensional structure, ROAD-EnKFs achieve higher accuracy at lower computational cost compared to existing methods. If such structure is not expressed in the latent state dynamics, ROAD-EnKFs achieve similar accuracy at lower cost, making them a promising approach for surrogate state reconstruction and forecasting.

Mathematics↗

Workflows for Science: A comprehensive guide for ensemble workflow tools usage with applications on OLCF systems

The growing demand for robust computational and workflow environments for scientific applications and user communities at the Oak Ridge Leadership Computing Facility (OLCF) has prompted collaboration with ensemble tools development teams and facility users to produce this technical paper. We connect science applications to the RADICAL-Pilot (RP) workflow tool to execute ensemble instantiations using the Frontier supercomputer. The documented installation, usage, and execution demonstrates how RP streamlines scientific workflows at OLCF. We outline the specific steps OLCF users can follow to integrate this tool with their applications and advance their research. This document stands as a comprehensive guide to OLCF users of ensemble workflow tools with examples on real applications using the Frontier supercomputer.

97 MATHEMATICS AND COMPUTING↗

Embedded ensemble propagation for improving performance, portability, and scalability of uncertainty quantification on emerging computational architectures

In this study, quantifying simulation uncertainties is a critical component of rigorous predictive simulation. A key component of this is forward propagation of uncertainties in simulation input data to output quantities of interest. Typical approaches involve repeated sampling of the simulation over the uncertain input data, and can require numerous samples when accurately propagating uncertainties from large numbers of sources. Often simulation processes from sample to sample are similar and much of the data generated from each sample evaluation could be reused. We explore a new method for implementing sampling methods that simultaneously propagates groups of samples together in an embedded fashion, which we call embedded ensemble propagation. We show how this approach takes advantage of properties of modern computer architectures to improve performance by enabling reuse between samples, reducing memory bandwidth requirements, improving memory access patterns, improving opportunities for fine-grained parallelization, and reducing communication costs. We describe a software technique for implementing embedded ensemble propagation based on the use of C++ templates and describe its integration with various scientific computing libraries within Trilinos. We demonstrate improved performance, portability and scalability for the approach applied to the simulation of partial differential equations on a variety of CPU, GPU, and accelerator architectures, including up to 131,072 cores on a Cray XK7 (Titan).

97 MATHEMATICS AND COMPUTING↗

A Machine Learning Framework for Modeling Ensemble Properties of Atomically Disordered Materials

Atomic disorder can strongly influence material properties such as charge transport, optical response, and catalytic activity. However, efficiently modeling these disorder effects remains challenging for first-principles methods due to the cost of sampling large configurational spaces and computing complex physical quantities. Recent advances of machine learning techniques, particularly graph neural networks (GNNs), has enabled the efficient and accurate predictions of complex material properties, offering promising tools for studying disordered systems. In this work, we present a general machine-learning-assisted computational framework that integrates equivariant GNNs with Monte Carlo simulations to compute the thermodynamic and ensemble-averaged functional properties of disordered materials. Using the surface-termination-disordered MXene monolayer Ti 3 C 2 T 2–x as a representative system, we find that electrical conductivity exhibits an emergent peak near the order–disorder phase transition temperature due to the interplay between electron scattering and doping. In contrast, optical conductivity remains largely insensitive to local atomic disorder and reflects the global surface chemical composition. These results highlight the role of atomic disorder in affecting material properties and demonstrate the potential of our approach for statistically modeling disorder effects in a wide range of materials such as high-entropy alloys and spin liquids.

MXene↗

Combining the ensemble and Franck-Condon approaches for calculating spectral shapes of molecules in solution

The correct treatment of vibronic effects is vital for the modeling of absorption spectra of many solvated dyes. Vibronic spectra for small dyes in solution can be easily computed within the Franck-Condon approximation using an implicit solvent model. However, implicit solvent models neglect specific solute-solvent interactions on the electronic excited state. On the other hand, a straightforward way to account for solute-solvent interactions and temperature-dependent broadening is by computing vertical excitation energies obtained from an ensemble of solute-solvent conformations. Ensemble approaches usually do not account for vibronic transitions and thus often produce spectral shapes in poor agreement with experiment. Here we address these shortcomings by combining zero-temperature vibronic fine structure with vertical excitations computed for a room-temperature ensemble of solute-solvent configurations. In this combined approach, all temperature-dependent broadening is treated classically through the sampling of configurations and quantum mechanical vibronic contributions are included as a zero-temperature correction to each vertical transition. In our calculation of the vertical excitations, significant regions of the solvent environment are treated fully quantum mechanically to account for solute-solvent polarization and charge-transfer. For the Franck-Condon calculations, a small amount of frozen explicit solvent is considered in order to capture solvent effects on the vibronic shape function. We test the proposed method by comparing calculated and experimental absorption spectra of Nile red and the green fluorescent protein chromophore in polar and non-polar solvents. For systems with strong solute-solvent interactions, the combined approach yields significant improvements over the ensemble approach. For systems with weak to moderate solute-solvent interactions, both the high-energy vibronic tail and the width of the spectra are in excellent agreement with experiments

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Computationally efficient implementation of combustion chemistry in parallel PDF calculations

In parallel calculations of combustion processes with realistic chemistry, the serial in situ adaptive tabulation (ISAT) algorithm [S.B. Pope, Computationally efficient implementation of combustion chemistry using in situ adaptive tabulation, Combustion Theory and Modelling, 1 (1997) 41-63; L. Lu, S.B. Pope, An improved algorithm for in situ adaptive tabulation, Journal of Computational Physics 228 (2009) 361-386] substantially speeds up the chemistry calculations on each processor. To improve the parallel efficiency of large ensembles of such calculations in parallel computations, in this work, the ISAT algorithm is extended to the multi-processor environment, with the aim of minimizing the wall clock time required for the whole ensemble. Parallel ISAT strategies are developed by combining the existing serial ISAT algorithm with different distribution strategies, namely purely local processing (PLP), uniformly random distribution (URAN), and preferential distribution (PREF). The distribution strategies enable the queued load redistribution of chemistry calculations among processors using message passing. They are implemented in the software x2f{sub m}pi, which is a Fortran 95 library for facilitating many parallel evaluations of a general vector function. The relative performance of the parallel ISAT strategies is investigated in different computational regimes via the PDF calculations of multiple partially stirred reactors burning methane/air mixtures. The results show that the performance of ISAT with a fixed distribution strategy strongly depends on certain computational regimes, based on how much memory is available and how much overlap exists between tabulated information on different processors. No one fixed strategy consistently achieves good performance in all the regimes. Therefore, an adaptive distribution strategy, which blends PLP, URAN and PREF, is devised and implemented. It yields consistently good performance in all regimes. In the adaptive parallel ISAT strategy, the type and extent of redistribution is determined 'on the fly' based on the prediction of future simulation time. Compared to the PLP/ISAT strategy where chemistry calculations are essentially serial, a speed-up factor of up to 30 is achieved. The study also demonstrates that the adaptive strategy has acceptable parallel scalability.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Calculating adsorption isotherms using the two-phase thermodynamic method and molecular dynamics simulations

We describe the calculation of adsorption isotherms from molecular dynamics simulations based on the two-phase thermodynamic (2PT) model. The 2PT model developed for bulk fluid phases treats the gas-like components as hard spheres (HSs), which correctly recovers the limiting behaviors of unconfined fluids. We showed that this treatment, however, does not always lead to the correct zero-loading behavior in strongly confining systems. For methane adsorption into zeolite MFI, the HS reference state underestimates entropy by up to 20% at low loadings and leads to an order-of-magnitude increase in the adsorption onset pressure. To fix these issues, we propose the use of ideal adsorbed gas (IAG) as the gas reference model, the properties of which can be computed using the Widom insertion method on an empty adsorbent. We further describe three routes to compute adsorption isotherms from the Helmholtz free energy at different loadings. Comparing against established Monte Carlo (MC) methods, we found that the adsorption isotherms obtained using the IAG reference state agrees to within 40%, which corresponds to deviations of <5% in adsorption free energy. The isotherms calculated using the HS reference state underestimate the adsorption uptake at low to medium loadings in strongly confining systems, but its accuracy improves at higher loadings and as the pore size increases relative to the sorbate diameter. The methods described here provide an alternative approach for computing adsorption isotherms when MC simulations in an open ensemble are undesirable and enable a direct comparison of computed adsorption thermodynamics with experiments.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Analysis of Filesystem Utilization by the ?Ensemble of Models? Approach (U)

In order to execute Uncertainty Quantification (UQ) studies, the number of reads placed on the filesystem will increase. This document works through the file I/O for a climatology UQ study. The utilization of the filesystem for the above discussed will be similar for ICF and Stockpile Stewardship applications. The current state of the art for the quantification of uncertainty of a multi-physics simulation code is the utilization of an ensemble of models approach. As an overview of the ensemble of models approach, a set of uncertain input parameters is identified along with an identified set of observational and output parameters. The model is computed n number of times with each ensemble simulation using a unique set of parametric combinations of input parameters thereby creating an ensemble of simulations. Response surface models (also known as statistical emulator models, surrogate models, or meta models) are trained using the ensemble results. The response models are then convolved with observational data to further constrain input parameters and to create uncertainty bounds on the model outputs. Using the Community Climate System Model (CCSM) specifically the atmospheric component of CCSM, the Community Atmospheric Model (CAM), as the model of interest, this document provides a rough model of the demands on a filesystem that is needed to execute a UQ study on CAM. Each ensemble simulation consists of 12 simulation years and uses 384 processors on the Atlas machine. The LLNL UQ Pipeline is LLNL's standard tool to execute UQ studies. The UQ Pipeline possesses the capability to execute the ensemble simulations on LLNL's diverse set of HPC environments, produce response models, generate uncertainty bounds, and analyze the results. The process executing the LLNL UQ Pipeline is run on a different compute node from the set of concurrent, executing ensemble simulations.

99 GENERAL AND MISCELLANEOUS↗

Reducing the complexity of finite-temperature auxiliary-field quantum Monte Carlo

The auxiliary-field quantum Monte Carlo (AFMC) method is a powerful and widely used technique for ground-state and finite-temperature simulations of quantum many-body systems. Here we introduce several algorithmic improvements for finite-temperature AFMC calculations of dilute fermionic systems that reduce the computational complexity of most parts of the algorithm. This is principally achieved by reducing the number of single-particle states that contribute at each configuration of the auxiliary fields to a number that is of the order of the number of fermions. Our methods are applicable for both the canonical and grand-canonical ensembles. We demonstrate the reduced computational complexity of the methods for the homogeneous unitary Fermi gas.

67.85.Lm↗

Portable, heterogeneous ensemble workflows at scale using libEnsemble

libEnsemble is a Python-based toolkit for running dynamic ensembles, developed as part of the DOE Exascale Computing Project. The toolkit utilizes a unique generator–simulator–allocator paradigm, where generators produce input for simulators, simulators evaluate those inputs, and allocators decide whether and when a simulator or generator should be called. The generator steers the ensemble based on simulation results. Generators may, for example, apply methods for numerical optimization, machine learning, or statistical calibration. libEnsemble communicates between a manager and workers. Flexibility is provided through multiple manager–worker communication substrates each of which has different benefits. These include Python’s multiprocessing, mpi4py, and TCP. Multisite ensembles are supported using Balsam or Globus Compute. We overview the unique characteristics of libEnsemble as well as current and potential interoperability with other packages in the workflow ecosystem. We highlight libEnsemble’s dynamic resource features: libEnsemble can detect system resources, such as available nodes, cores, and GPUs, and assign these in a portable way. These features allow users to specify the number of processors and GPUs required for each simulation; and resources will be automatically assigned on a wide range of systems, including Frontier, Aurora, and Perlmutter. Such ensembles can include multiple simulation types, some using GPUs and others using only CPUs, sharing nodes for maximum efficiency. We also describe the benefits of libEnsemble’s generator–simulator coupling, which easily exposes to the user the ability to cancel, and portably kill, running simulations based on models that are updated with intermediate simulation output. We demonstrate libEnsemble’s capabilities, scalability, and scientific impact via a Gaussian process surrogate training problem for the longitudinal density profile at the exit of a plasma accelerator stage. In conclusion, the study uses gpCAM for the surrogate model and employs either Wake-T or WarpX simulations, highlighting efficient use of resources that can easily extend to exascale.

Dynamic ensembles↗

Estimating Uncertainty in Simulated ENSO Statistics

Abstract Large ensembles of model simulations are frequently used to reduce the impact of internal variability when evaluating climate models and assessing climate change induced trends. However, the optimal number of ensemble members required to distinguish model biases and climate change signals from internal variability varies across models and metrics. Here we analyze the mean, variance and skewness of precipitation and sea surface temperature in the eastern equatorial Pacific region often used to describe the El Niño–Southern Oscillation (ENSO), obtained from large ensembles of Coupled model intercomparison project phase 6 climate simulations. Leveraging established statistical theory, we develop and assess equations to estimate, a priori, the ensemble size or simulation length required to limit sampling‐based uncertainties in ENSO statistics to within a desired tolerance. Our results confirm that the uncertainty of these statistics decreases with the square root of the time series length and/or ensemble size. Moreover, we demonstrate that uncertainties of these statistics are generally comparable when computed using either pre‐industrial control or historical runs. This suggests that pre‐industrial runs can sometimes be used to estimate the expected uncertainty of statistics computed from an existing historical member or ensemble, and the number of simulation years (run duration and/or ensemble size) required to adequately characterize the statistic. This advance allows us to use existing simulations (e.g., control runs that are performed during model development) to design ensembles that can sufficiently limit diagnostic uncertainties arising from simulated internal variability. These results may well be applicable to variables and regions beyond ENSO.

54 ENVIRONMENTAL SCIENCES↗

Short-wavelength plasma turbulence and temperature anisotropy instabilities: Recent computational progress

Plasma turbulence consists of an ensemble of enhanced, broadband electromagnetic fluctuations, typically driven by multi-wave interactions which transfer energy in wavevector space via non- linear cascade processes. In addition, temperature anisotropy instabilities in collisionless plasmas are driven by quasi-linear wave–particle interactions which transfer particle kinetic energy to field fluctuation energy; the resulting enhanced fluctuations are typically narrowband in wavevector magnitude and direction. Whatever their sources, short-wavelength fluctuations are those at which charged particle kinetic, that is, velocity-space, properties are important; these are generally wavelengths of the order of or shorter than the ion inertial length or the thermal ion gyroradius. The purpose of this review is to summarize and interpret recent computational results concerning short-wavelength plasma turbulence, short-wavelength temperature anisotropy instabilities and relationships between the two phenomena.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Accurate predictions from small boxes: variance suppression via the Zel'dovich approximation

Simulations have become an indispensable tool for accurate modelling of observables measured in galaxy surveys, but can be expensive if very large dynamic range in scale is required. We describe how to combine Lagrangian perturbation theory models with N-body simulations to reduce the effects of finite computational volume in the prediction of ensemble average properties in the simulations within the context of control variates. In particular we use the fact that Zel'dovich displacements, computed during initial condition generation for any simulation, correlate strongly with the final density field. Since all the correlators of biased tracers can be computed with arbitrary precision for these displacements, pairing the Zel'dovich `simulation' with the N-body realization allows hundredfold reductions in sample variance for power spectrum or correlation function estimation. Additionally, Zel'dovich control variates can accurately extend matter or tracer field emulators to larger scales than previously possible, as well as improving measurements of statistics in simulations which are inherently limited to small volumes, such as hydrodynamical simulations of galaxy formation and reionization.

79 ASTRONOMY AND ASTROPHYSICS↗

Orchestrating Distributed Resource Ensembles for Petascale Science

Distributed, data-intensive computational science applications of interest to DOE scientific com- munities move large amounts of data for experiment data management, distributed analysis steps, remote visualization, and accessing scientific instruments. These applications need to orchestrate ensembles of resources from multiple resource pools and interconnect them with high-capacity multi- layered networks across multiple domains. It is highly desirable that mechanisms are designed that provide this type of resource provisioning capability to a broad class of applications. It is also important to have coherent monitoring capabilities for such complex distributed environments. In this project, we addressed these problems by designing an abstract API, enabled by novel semantic resource descriptions, for provisioning complex and heterogeneous resources from multiple providers using their native provisioning mechanisms and control planes: computational, storage, and multi-layered high-speed network domains. We used an extensible resource representation based on semantic web technologies to afford maximum flexibility to applications in specifying their needs. We evaluated the effectiveness of provisioning using representative data-intensive ap- plications. We also developed mechanisms for providing feedback about resource performance to the application, to enable closed-loop feedback control and dynamic adjustments to resource allo- cations (elasticity). This was enabled through development of a novel persistent query framework that consumes disparate sources of monitoring data, including perfSONAR, and provides scalable distribution of asynchronous notifications.

97 MATHEMATICS AND COMPUTING↗

Manipulation of self-organized multi-vortical states in active magnetic roller suspensions

Ensembles of active magnetic colloids demonstrate complex collective behavior and self-organization when driven out of equilibrium by external magnetic fields. The interplay between magnetic and hydrodynamic interactions often result in spontaneous self-assembly and the emergence of localized vortices in ensembles of ferromagnetic rollers. We report experimental and computational study of the self-organized multi-vortical state in ensembles of magnetic rollers under the imposed external confining potential realized by patterned substrates. We explore the behavior of the system on a substrate patterned with shallow wells ordered in a lattice with square symmetry and lattice spacing smaller than the wells’ diameter. Here we demonstrate, that ensembles of active magnetic rollers evolve in response to changes in the geometry of the confining lattice from a globally correlated state characterized by anti-ferromagnetic vortex ordering to a state with rapidly changing particle flows and self-organized vortical states with a lattice constant larger than one of the confining pattern. Our experimental observations are accompanied by numerical simulations based on phenomenological coarse grained particle dynamics coupled to Navier–Stokes hydrodynamics in shallow water approximation. The reported results provide insights into the collective behavior of active magnetic rollers under confinement and suggest strategies for the manipulation of collective dynamic states in active magnetic liquids.

36 MATERIALS SCIENCE↗

Validation of macromolecular flexibility in solution by small-angle X-ray scattering (SAXS)

The dynamics of macromolecular conformations are critical to the action of cellular networks. Solution X-ray scattering studies, in combination with macromolecular X-ray crystallography (MX) and nuclear magnetic resonance (NMR), strive to determine complete and accurate states of macromolecules, providing novel insights describing allosteric mechanisms, supramolecular complexes, and dynamic molecular machines. This review addresses theoretical and practical concepts, concerns, and considerations for using these techniques in conjunction with computational methods to productively combine solution-scattering data with high-resolution structures. I discuss the principal means of direct identification of macromolecular flexibility from SAXS data followed by critical concerns about the methods used to calculate theoretical SAXS profiles from high-resolution structures. The SAXS profile is a direct interrogation of the thermodynamic ensemble and techniques such as, for example, minimal ensemble search (MES), enhance interpretation of SAXS experiments by describing the SAXS profiles as population-weighted thermodynamic ensembles. I discuss recent developments in computational techniques used for conformational sampling, and how these techniques provide a basis for assessing the level of the flexibility within a sample. Although these approaches sacrifice atomic detail, the knowledge gained from ensemble analysis is often appropriate for developing hypotheses and guiding biochemical experiments. Examples of the use of SAXS and combined approaches with X-ray crystallography, NMR, and computational methods to characterize dynamic assemblies are presented.

59 BASIC BIOLOGICAL SCIENCES↗