Search NASA⌕ Search

Engineering topics

Reynolds, Daniel R.

Publications and source records attributed to Reynolds, Daniel R..

SUNDIALS time integrators for exascale applications with many independent systems of ordinary differential equations

Many complex systems can be accurately modeled as a set of coupled time-dependent partial differential equations (PDEs). However, solving such equations can be prohibitively expensive, easily taxing the world’s largest supercomputers. One pragmatic strategy for attacking such problems is to split the PDEs into components that can more easily be solved in isolation. This operator splitting approach is used ubiquitously across scientific domains, and in many cases leads to a set of ordinary differential equations (ODEs) that need to be solved as part of a larger “outer-loop” time-stepping approach. The SUNDIALS library provides a plethora of robust time integration algorithms for solving ODEs, and the U.S. Department of Energy Exascale Computing Project (ECP) has supported its extension to applications on exascale-capable computing hardware. In this paper, we highlight some SUNDIALS capabilities and its deployment in combustion and cosmology application codes (Pele and Nyx, respectively) where operator splitting gives rise to numerous, small ODE systems that must be solved concurrently.

97 MATHEMATICS AND COMPUTING↗

Implicit–explicit multirate infinitesimal stage-restart methods

Implicit–Explicit (IMEX) methods are flexible numerical time integration methods which solve an initial-value problem (IVP) that is split into stiff and nonstiff processes with the goal of lower computational costs than a purely implicit or explicit approach. A complementary form of flexible IVP solvers are multirate infinitesimal methods for problems split into fast- and slow-changing dynamics, that solve a multirate IVP by evolving a sequence of “fast” IVPs using any suitably accurate algorithm. This article introduces a new class of high-order implicit–explicit multirate methods that are designed for multirate IVPs in which the slow-changing dynamics are further split in an IMEX fashion. This new class, which we call implicit–explicit multirate infinitesimal stage-restart (IMEX-MRI-SR), both improves upon the previous implicit–explicit multirate infinitesimal generalized-structure additive Runge Kutta (IMEX-MRI-GARK) methods by allowing for far easier creation of new embedded methods, and extends multirate exponential Runge Kutta (MERK) methods by allowing the fast-changing dynamics to be nonlinear and the methods to be implicit. We leverage GARK theory to derive conditions for orders of accuracy up to four, and we provide second- and third-order accurate example methods, which are the first known embedded MRI methods with IMEX structure. We then perform numerical simulations demonstrating convergence rates and computational performance in both fixed-step and adaptive-step settings.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Performance of explicit and IMEX MRI multirate methods on complex reactive flow problems within modern parallel adaptive structured grid frameworks

Large-scale multiphysics simulations are computationally challenging due to the coupling of multiple processes with widely disparate time scales. The advent of exascale computing systems exacerbates these challenges since these systems enable ever-increasing size and complexity. In recent years, there has been renewed interest in developing multirate methods as a means to handle the large range of time scales, as these methods may afford greater accuracy and efficiency than more traditional approaches of using implicit-explicit (IMEX) and low-order operator splitting schemes. However, to date there have been few performance studies that compare different classes of multirate integrators on complex application problems. In this work, we study the performance of several newly developed multirate infinitesimal (MRI) methods, implemented in the SUNDIALS solver package, on two reacting flow model problems built on structured mesh frameworks. The first model revisits prior work on a compressible reacting flow problem with complex chemistry that is implemented using BoxLib but where we now include comparisons between a new explicit MRI scheme with the multirate spectral deferred correction (SDC) methods in the original paper. The second problem uses the same complex chemistry as the first problem, combined with a simplified flow model, but runs at a large spatial scale where explicit methods become infeasible due to stability constraints. Two recently developed IMEX MRI multirate methods are tested. These methods rely on advanced features of the AMReX framework on which the model is built, such as multilevel grids and multilevel preconditioners. The results from these two problems show that MRI multirate methods can offer significant performance benefits on complex multiphysics application problems and that these methods may be combined with advanced spatial discretization to compound the advantages of both.

97 MATHEMATICS AND COMPUTING↗

ARKODE: A Flexible IVP Solver Infrastructure for One-step Methods

We describe the ARKODE library of one-step time integration methods for ordinary differential equation (ODE) initial-value problems (IVPs). In addition to providing standard explicit and diagonally implicit Runge–Kutta methods, ARKODE supports one-step methods designed to treat additive splittings of the IVP, including implicit-explicit (ImEx) additive Runge–Kutta methods and multirate infinitesimal (MRI) methods. We present the role of ARKODE within the SUNDIALS suite of time integration and nonlinear solver libraries, the core ARKODE infrastructure for utilities common to large classes of one-step methods, as well as its use of “time stepper” modules enabling easy incorporation of novel algorithms into the library. Numerical results show example problems of increasing complexity, highlighting the algorithmic flexibility afforded through this infrastructure, and include a larger multiphysics application leveraging multiple algorithmic features from ARKODE and SUNDIALS.

97 MATHEMATICS AND COMPUTING↗

Adaptive Time Step Control for Multirate Infinitesimal Methods

Multirate methods have been used for decades to temporally evolve initial-value problems in which different components evolve on distinct time scales, and thus use of different step sizes for these components can result in increased computational efficiency. Generally, such methods select these different step sizes based on experimentation or stability considerations. For problems that evolve on a single time scale, adaptivity approaches that strive to control local temporal error are widely used to achieve numerical results of a desired accuracy with minimal computational effort, while alleviating the need for manual experimentation with different time step sizes. However, there is a notable gap in the publication record on the development of adaptive time step controllers for multirate methods. In this paper, we extend the single-rate controller work of Gustafsson [ACM Trans. Math. Software, 20 (1994), pp. 496-517] to the multirate method setting. Specifically, we develop controllers based on polynomial approximations to the principal error functions for both the "fast" and "slow" time scales within multirate infinitesimal (MRI) methods. We additionally investigate a variety of approaches for estimating the errors arising from each time scale within MRI methods. We then numerically evaluate the proposed multirate controllers and error estimation strategies on a range of multirate test problems, comparing their performance against an estimated optimal performance. Through this work, we combine the most performant of these approaches to arrive at a set of multirate adaptive time step controllers that robustly achieve desired solution accuracy with minimal computational effort.

97 MATHEMATICS AND COMPUTING↗

Multirate Exponential Rosenbrock Methods

In this paper we propose a novel class of methods for high-order accurate integration of multirate systems of ordinary differential equation initial-value problems. The proposed methods construct multirate schemes by approximating the action of matrix φ functions within explicit exponential Rosenbrock (ExpRB) methods, thereby called multirate ExpRB (MERB) methods. They consist of the solution to a sequence of modified “fast” initial-value problems, which may themselves be approximated through subcycling any desired initial-value problem solver. In addition to proving how to construct MERB methods from certain classes of ExpRB methods, we provide rigorous convergence analysis of these methods and derive efficient MERB schemes of orders 2 through 6 (the highest-order infinitesimal multirate methods to date). Lastly, we then present numerical simulations to confirm these theoretical convergence rates and to compare the efficiency of MERB methods against other recently introduced high-order multirate methods.

97 MATHEMATICS AND COMPUTING↗

Enabling New Flexibility in the SUNDIALS Suite of Nonlinear and Differential/Algebraic Equation Solvers

In recent years, the SUite of Nonlinear and DIfferential/ALgebraic equation Solvers (SUNDIALS) has been redesigned to better enable the use of application-specific and third-party algebraic solvers and data structures. Throughout this work, we have adhered to specific guiding principles that minimized the impact to current users while providing maximum flexibility for later evolution of solvers and data structures. The redesign was done through the addition of new linear and nonlinear solvers classes, enhancements to the vector class, and the creation of modern Fortran interfaces. The vast majority of this work has been performed “behind-the-scenes,” with minimal changes to the user interface and no reduction in solver capabilities or performance. These changes allow SUNDIALS users to more easily utilize external solver libraries and create highly customized solvers, enabling greater flexibility on extreme-scale, heterogeneous computational architectures.

97 MATHEMATICS AND COMPUTING↗

Implicit-Explicit Multirate Infinitesimal GARK Methods

This work focuses on the development of a new class of high-order accurate methods for multirate time integration of systems of ordinary differential equations. Unlike other recent work in this area, the proposed methods support mixed implicit-explicit (IMEX) treatment of the slow time scale. In addition to allowing this slow time scale flexibility, the proposed methods utilize a so-called infinitesimal formulation for the fast time scale through definition of a sequence of modified “fast" initial-value problems that may be solved using any viable algorithm. We name the proposed class as implicit-explicit multirate infinitesimal generalized-structure additive Runge--Kutta (IMEX-MRI-GARK) methods. In addition to defining these methods, we prove that they may be viewed as specific instances of GARK methods and derive a set of order conditions on the IMEX-MRI-GARK coefficients to guarantee both third and fourth order accuracy for the overall multirate method. Additionally, we provide three specific IMEX-MRI-GARK methods, two of order three and one of order four. We conclude with numerical simulations on two multirate test problems, demonstrating the methods' predicted convergence rates and comparing their efficiency against both legacy IMEX multirate schemes and recent third and fourth order implicit MRI-GARK methods.

97 MATHEMATICS AND COMPUTING↗

Evaluation of Implicit‐Explicit Additive Runge‐Kutta Integrators for the HOMME‐NH Dynamical Core

Abstract The nonhydrostatic High‐Order Method Modeling Environment (HOMME‐NH) atmospheric dynamical core supports acoustic waves that propagate significantly faster than the advective wind speed, thus greatly limiting the time step size that can be used with standard explicit time integration methods. Resolving acoustic waves is unnecessary for accurate climate and weather prediction. This numerical stiffness is addressed herein by considering implicit‐explicit additive Runge‐Kutta (ARK IMEX) methods that can treat the acoustic waves in a stable manner without requiring implicit treatment of nonstiff modes. Various ARK IMEX methods are evaluated for their efficiency in producing accurate solutions, ability to take large time step sizes, and sensitivity to grid cell length ratio. Both the gravity wave test and baroclinic instability test from the 2012 Dynamical Core Model Intercomparison Project are used to recommend 5 of the 27 ARK IMEX methods tested for use in HOMME‐NH.

97 MATHEMATICS AND COMPUTING↗

SUNDIALS Multiphysics+MPIManyVector Performance Testing

In this report we document performance test results on a SUNDIALS-based multiphysics demonstration application. We aim to assess the large-scale parallel performance of new capabilities that have been added to the SUNDIALS suite of time integrators and nonlinear solvers in recent years under funding from both the Exascale Computing Project (ECP) and the Scientific Discovery through Advanced Scientific (SciDAC) program, specifically: 1. SUNDIALS’ new MPIManyVector module, that allows extreme flexibility in how a solution "vector" is staged on computational resources. 2. ARKode’s new multirate integration module, MRIStep, allowing high-order accurate calculations that subcycle "fast" processes within "slow" ones. 3. SUNDIALS’ new flexible linear solver interfaces, that allow streamlined specification of problem-specific linear solvers. 4. SUNDIALS’ new N_Vector additions of "fused" vector operations (to increase arithmetic intensity) and separation of reduction operations into "local" and "global" versions (to reduce latency by combining multiple reductions into a single MPI_Allreduce call). We anticipate that subsequent reports will extend this work to investigate a variety of other new features, including SUNDIALS’ generic SUNNonlinearSolver interface and accelerator-enabled N Vector modules, and upcoming MRIStep extensions to support custom "fast" integrators (that leverage problem structure) and IMEX integration of the "slow" time scale (to add diffusion).

97 MATHEMATICS AND COMPUTING↗

Filters for Improvement of Multiscale Data from Atomistic Simulations

Multiscale computational models strive to produce accurate and efficient numerical simulations of systems involving interactions across multiple spatial and temporal scales that typically differ by several orders of magnitude. Some such models utilize a hybrid continuum-atomistic approach combining continuum approximations with first-principles-based atomistic models to capture multiscale behavior. By following the heterogeneous multiscale method framework for developing multiscale computational models, unknown continuum scale data can be computed from an atomistic model. Concurrently coupling the two models requires performing numerous atomistic simulations which can dominate the computational cost of the method. Furthermore, when the resulting continuum data is noisy due to sampling error, stochasticity in the model, or randomness in the initial conditions, filtering can result in significant accuracy gains in the computed multiscale data without increasing the size or duration of the atomistic simulations. In this work, we demonstrate the effectiveness of spectral filtering for increasing the accuracy of noisy multiscale data obtained from atomistic simulations. Moreover, we present a robust and automatic method for closely approximating the optimum level of filtering in the case of additive white noise. By improving the accuracy of this filtered simulation data, it leads to a dramatic computational savings by allowing for shorter and smaller atomistic simulations to achieve the same desired multiscale simulation precision.

97 MATHEMATICS AND COMPUTING↗

Fully coupled simulation of cosmic reionization. I. numerical methods and tests

Here, we describe an extension of the Enzo code to enable fully coupled radiation hydrodynamical simulation of inhomogeneous reionization in large similar to(100 Mpc)(3) cosmological volumes with thousands to millions of point sources. We solve all dynamical, radiative transfer, thermal, and ionization processes self-consistently on the same mesh, as opposed to a postprocessing approach which coarse-grains the radiative transfer. But, we employ a simple subgrid model for star formation which we calibrate to observations. The numerical method presented is a modification of an earlier method presented in Reynolds et al. differing principally in the operator splitting algorithm we use to advance the system of equations. Radiation transport is done in the gray flux-limited diffusion (FLD) approximation, which is solved by implicit time integration split off from the gas energy and ionization equations, which are solved separately. This results in a faster and more robust scheme for cosmological applications compared to the earlier method. The FLD equation is solved using the hypre optimally scalable geometric multigrid solver from LLNL. By treating the ionizing radiation as a grid field as opposed to rays, our method is scalable with respect to the number of ionizing sources, limited only by the parallel scaling properties of the radiation solver. We test the speed and accuracy of our approach on a number of standard verification and validation tests. We show by direct comparison with Enzo's adaptive ray tracing method Moray that the well-known inability of FLD to cast a shadow behind opaque clouds has a minor effect on the evolution of ionized volume and mass fractions in a reionization simulation validation test. Finally, we illustrate an application of our method to the problem of inhomogeneous reionization in a 80 Mpc comoving box resolved with 3200(3) Eulerian grid cells and dark matter particles.

79 ASTRONOMY AND ASTROPHYSICS↗

Fully coupled simulation of cosmic reionization. II. Recombinations, clumping factors, and the photon budget for reionization

We use a fully coupled cosmological simulation including dark matter dynamics, multispecies hydrodynamics, nonequilibrium chemical ionization, flux-limited diffusion radiation transport, and a parameterized model of star formation and feedback (thermal and radiative) to investigate the epoch of hydrogen reionization in detail. In this paper, the first of several application papers, we investigate the mechanics of reionization from stellar sources forming in high-z galaxies, the utility of various formulations for the gas clumping factor on accurately estimating the effective recombination time in the intergalactic medium (IGM), and the photon budget required to achieve reionization. We also test the accuracy of the static and time-dependent models of Madau et al. as predictors of reionization completion/maintenance. We simulate a WMAP7 ΛCDM cosmological model in a 20 comoving Mpc cube, resolved with 800{sup 3} uniform fluid cells and dark matter particles. By tuning our star formation recipe to approximately match the observed high-redshift star formation rate density and galaxy luminosity function, we have created a fully coupled radiation hydrodynamical realization of hydrogen reionization, which begins to ionize at z ≈ 10 and is completed at z ≈ 5.8 without further tuning. We find that roughly two ionizing photons per H atom are required to convert the neutral IGM to a highly ionized state. After reionization concludes, we find that the quantity n-dot{sub ion}×(1 Gyr)/n{sub H} is ∼9 at z = 5, in rough agreement with measurements of the ionizing emissivity by Becker and Bolton. The complicated events during reionization that lead to this number can be generally described as inside-out, but in reality, the narrative depends on the level of ionization of the gas one attributes as being ionized. We find that the formula for the ionizing photon production rate needed to maintain the IGM in an ionized state derived by Madau et al. should not be used to predict the epoch of reionization completion because it ignores history-dependent terms in the global ionization balance which are not ignorable. We find that the time-dependent model for the ionized volume fraction Q{sub HII} is more predictive, but overestimates the redshift of reionization completion z{sub reion} by Δz ≈ 1. We propose a revised formulation of the time-dependent model that agrees with our simulation to a few percent accuracy. Finally, we use our simulation data to measure the absorption of ionizing radiation due to circumgalactic gas resolved on our mesh and find f-bar{sub esc}(CGM)≈0.7.

79 ASTRONOMY AND ASTROPHYSICS↗

ENZO: AN ADAPTIVE MESH REFINEMENT CODE FOR ASTROPHYSICS

This paper describes the open-source code Enzo, which uses block-structured adaptive mesh refinement to provide high spatial and temporal resolution for modeling astrophysical fluid flows. The code is Cartesian, can be run in one, two, and three dimensions, and supports a wide variety of physics including hydrodynamics, ideal and non-ideal magnetohydrodynamics, N-body dynamics (and, more broadly, self-gravity of fluids and particles), primordial gas chemistry, optically thin radiative cooling of primordial and metal-enriched plasmas (as well as some optically-thick cooling models), radiation transport, cosmological expansion, and models for star formation and feedback in a cosmological context. In addition to explaining the algorithms implemented, we present solutions for a wide range of test problems, demonstrate the code's parallel performance, and discuss the Enzo collaboration's code development methodology.

79 ASTRONOMY AND ASTROPHYSICS↗

Vibration damping and heat transfer using material phase changes

A method and apparatus wherein phase changes in a material can dampen vibrational energy, dampen noise and facilitate heat transfer. One embodiment includes a method for damping vibrational energy in a body. The method comprises attaching a material to the body, wherein the material comprises a substrate, a shape memory alloy layer, and a plurality of temperature change elements. The method further comprises sensing vibrations in the body. In addition, the method comprises indicating to at least a portion of the temperature change elements to provide a temperature change in the shape memory alloy layer, wherein the temperature change is sufficient to provide a phase change in at least a portion of the shape memory alloy layer, and further wherein the phase change consumes a sufficient amount of kinetic energy to dampen at least a portion of the vibrational energy in the body. In other embodiments, the shape memory alloy layer is a thin film. Additional embodiments include a sensor connected to the material.

Kloucek, Petr↗