Engineering topics

Emberson, J. D.

Publications and source records attributed to Emberson, J. D..

4 records

DOE OSTI2024

Advances in ArborX to support exascale applications

ArborX is a performance portable geometric search library developed as part of the Exascale Computing Project (ECP). In this paper, we explore a collaboration between ArborX and a cosmological simulation code HACC. Large cosmological simulations on exascale platforms encounter a bottleneck due to the in-situ analysis requirements of halo finding, a problem of identifying dense clusters of dark matter (halos). This problem is solved by using a density-based DBSCAN clustering algorithm. With each MPI rank handling hundreds of millions of particles, it is imperative for the DBSCAN implementation to be efficient. In addition, the requirement to support exascale supercomputers from different vendors necessitates performance portability of the algorithm. We describe how this challenge problem guided ArborX development, and enhanced the performance and the scope of the library. We explore the improvements in the basic algorithms for the underlying search index to improve the performance, and describe several implementations of DBSCAN in ArborX. Further, we report the history of the changes in ArborX and their effect on the time to solve a representative benchmark problem, as well as demonstrate the real world impact on production end-to-end cosmology simulations.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Improving initialization and evolution accuracy of cosmological neutrino simulations

Neutrino mass constraints are a primary focus of current and future large-scale structure (LSS) surveys. Non-linear LSS models rely heavily on cosmological simulations — the impact of massive neutrinos should therefore be included in these simulations in a realistic, computationally tractable, and controlled manner. Here, a recent proposal to reduce the related computational cost employs a symmetric neutrino momentum sampling strategy in the initial conditions. We implement a modified version of this strategy into the Hardware/Hybrid Accelerated Cosmology Code (HACC) and perform convergence tests on its internal parameters. We illustrate that this method can impart Script $\mathcal{O}$(1%) numerical artifacts on the total matter field on small scales, similar to previous findings, and present a method to remove these artifacts using Fourier-space filtering of the neutrino density field. Moreover, we show that the converged neutrino power spectrum does not follow linear theory predictions on relatively large scales at early times at the 15% level, prompting a more careful study of systematics in particle-based neutrino simulations. We also present an improved method for backscaling linear transfer functions for initial conditions in massive neutrino cosmologies that is based on achieving the same relative neutrino growth as computed with Boltzmann solvers. Our self-consistent backscaling method yields sub-percent accuracy in the total matter growth function. Comparisons for the non-linear power spectrum with the Mira-Titan emulator at a neutrino mass of m ν = 0.15 eV are in very good agreement with the expected level of errors in the emulator and in the direct N-body simulation.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2023

Numerical discreteness errors in multispecies cosmological N -body simulations

ABSTRACT We present a detailed analysis of numerical discreteness errors in two-species, gravity-only, cosmological simulations using the density power spectrum as a diagnostic probe. In a simple set-up where both species are initialized with the same total matter transfer function, biased growth of power forms on small scales when the solver force resolution is finer than the mean interparticle separation. The artificial bias is more severe when individual density and velocity transfer functions are applied. In particular, significant large-scale offsets in power are measured between simulations with conventional offset grid initial conditions when compared against converged high-resolution results where the force resolution scale is matched to the interparticle separation. These offsets persist even when the cosmology is chosen so that the two particle species have the same mass, indicating that the error is sourced from discreteness in the total matter field as opposed to unequal particle mass. We further investigate two mitigation strategies to address discreteness errors: the frozen potential method and softened interspecies short-range forces. The former evolves particles under the approximately ‘frozen’ total matter potential in linear theory at early times, while the latter filters cross-species gravitational interactions on small scales in low-density regions. By modelling closer to the continuum limit, both mitigation strategies demonstrate considerable reductions in large-scale power spectrum offsets.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

Modelling the Lyman-α forest with Eulerian and SPH hydrodynamical methods

ABSTRACT We compare two state-of-the-art numerical codes to study the overall accuracy in modelling the intergalactic medium and reproducing Lyman-α forest observables for DESI and high-resolution data sets. The codes employ different approaches to solving both gravity and modelling the gas hydrodynamics. The first code, Nyx, solves the Poisson equation using the Particle-Mesh (PM) method and the Euler equations using a finite-volume method. The second code, CRK-HACC , uses a Tree-PM method to solve for gravity, and an improved Lagrangian smoothed particle hydrodynamics (SPH) technique, where fluid elements are modelled with particles, to treat the intergalactic gas. We compare the convergence behaviour of the codes in flux statistics as well as the degree to which the codes agree in the converged limit. We find good agreement overall with differences being less than observational uncertainties, and a particularly notable ≲1 per cent agreement in the 1D flux power spectrum. This agreement was achieved by applying a tessellation methodology for reconstructing the density in CRK-HACC instead of using an SPH kernel as is standard practice. We show that use of the SPH kernel can lead to significant and unnecessary biases in flux statistics; this is especially prominent at high redshifts, z ∼ 5, as the Lyman-α forest mostly comes from lower-density regions that are intrinsically poorly sampled by SPH particles.

79 ASTRONOMY AND ASTROPHYSICS↗