Search NASA⌕ Search

Engineering topics

Pope, Adrian

Publications and source records attributed to Pope, Adrian.

At least 19 records

2023 Operational Assessment Report: Argonne Leadership Computing Facility

In 2004, the U.S. Department of Energy’s (DOE’s) Advanced Scientific Computing Research (ASCR) program established the Leadership Computing Facility (LCF) with a mission to provide the world’s most advanced computational resources to the open science community. The LCF is a huge investment in the nation’s scientific and technological future, inspired by a growing demand for large-scale computing and its impact on science and engineering. The LCF operates two world-class centers in support of open science at Argonne National Laboratory (Argonne) and at Oak Ridge National Laboratory (Oak Ridge) and deploys diverse machines that are among the most powerful systems in the world today. The LCF ranks among the top U.S. scientific facilities delivering impactful science. The work performed at these centers informs policy decisions and advances innovations in far-reaching areas such as energy assurance, ecological sustainability, and global security. The leadership-class systems at Argonne and Oak Ridge operate around the clock every day of the year. The high level of services these centers provide and the exceptional science they produce justify their existence to the DOE Office of Science and the U.S. Congress

97 MATHEMATICS AND COMPUTING↗

2022 Operational Assessment Report - Argonne Leadership Computing Facility

This Operational Assessment Report describes how the Argonne Leadership Computing Facility (ALCF) met or exceeded every one of its goals for the calendar year (CY) 2022. In CY 2022, ALCF operated Theta, an Intel-based Cray XC40 system (11.7 petaflops) augmented with 24 NVIDIA DGX A100-based nodes (3.9 petaflops), that supports diverse workloads, integrating data analytics with artificial intelligence (AI) training and learning in a single platform; and Polaris, a 44-petaflop AMD and NVIDIA-based HPE Apollo 6500 Gen10+ system that provides a powerful new platform to prepare applications and workloads for Aurora, Argonne National Laboratory’s (Argonne’s) upcoming Intel-Hewlett Packard Enterprise (HPE) exascale supercomputer.

97 MATHEMATICS AND COMPUTING↗

Galaxy Clustering in the Mira-Titan Universe. I. Emulators for the Redshift Space Galaxy Correlation Function and Galaxy–Galaxy Lensing

Abstract We construct accurate emulators for the projected and redshift space galaxy correlation functions and excess surface density as measured by galaxy–galaxy lensing, based on halo occupation distribution modeling. Using the complete Mira-Titan suite of 111 N -body simulations, our emulators vary over eight cosmological parameters and include the effects of neutrino mass and dynamical dark energy. We demonstrate that our emulators are sufficiently accurate for the analysis of the Baryon Oscillation Spectroscopic Survey DR12 CMASS galaxy sample over the range 0.5 ≤ r ≤ 50 h −1 Mpc. Furthermore, we show that our emulators are capable of recovering unbiased cosmological constraints from realistic mock catalogs over the same range. Our mock catalog tests show the efficacy of combining small-scale galaxy–galaxy lensing with redshift space clustering and that we can constrain the growth rate and σ 8 to 7% and 4.5%, respectively, for a CMASS-like sample using only the measurements covered by our emulator. With the inclusion of a cosmic microwave background prior on H 0 , this reduces to a 2% measurement of the growth rate.

79 ASTRONOMY AND ASTROPHYSICS↗

Farpoint: A High-resolution Cosmology Simulation at the Gigaparsec Scale

Abstract In this paper we introduce the Farpoint simulation, the latest member of the Hardware/Hybrid Accelerated Cosmology Code (HACC) gravity-only simulation family. The domain covers a volume of (1000 h −1 Mpc) 3 and evolves close to two trillion particles, corresponding to a mass resolution of m p ∼ 4.6 × 10 7 h −1 M ⊙ . These specifications enable comprehensive investigations of the galaxy–halo connection, capturing halos down to small masses. Further, the large volume resolves scales typical of modern surveys with good statistical coverage of high-mass halos. The simulation was carried out on the GPU-accelerated system Summit, one of the fastest supercomputers currently available. We provide specifics about the Farpoint run and present an initial set of results. The high mass resolution facilitates precise measurements of important global statistics, such as the halo concentration–mass relation and the correlation function down to small scales. Selected subsets of the simulation data products are publicly available via the HACC Simulation Data Portal.

79 ASTRONOMY AND ASTROPHYSICS↗

Enabling particle applications for exascale computing platforms

The Exascale Computing Project (ECP) is invested in co-design to assure that key applications are ready for exascale computing. Within ECP, the Co-design Center for Particle Applications (CoPA) is addressing challenges faced by particle-based applications across four “sub-motifs”: short-range particle–particle interactions (e.g., those which often dominate molecular dynamics (MD) and smoothed particle hydrodynamics (SPH) methods), long-range particle–particle interactions (e.g., electrostatic MD and gravitational N-body), particle-in-cell (PIC) methods, and linear-scaling electronic structure and quantum molecular dynamics (QMD) algorithms. Our crosscutting co-designed technologies fall into two categories: proxy applications (or “apps”) and libraries. Proxy apps are vehicles used to evaluate the viability of incorporating various types of algorithms, data structures, and architecture-specific optimizations and the associated trade-offs; examples include ExaMiniMD, CabanaMD, CabanaPIC, and ExaSP2. Libraries are modular instantiations that multiple applications can utilize or be built upon; CoPA has developed the Cabana particle library, PROGRESS/BML libraries for QMD, and the SWFFT and fftMPI parallel FFT libraries. Success is measured by identifiable “lessons learned” that are translated either directly into parent production application codes or into libraries, with demonstrated performance and/or productivity improvement. The libraries and their use in CoPA’s ECP application partner codes are also addressed.

97 MATHEMATICS AND COMPUTING↗

The Last Journey. I. An Extreme-scale Simulation on the Mira Supercomputer

The Last Journey is a large-volume, gravity-only, cosmological N-body simulation evolving more than 1.24 trillion particles in a periodic box with a side length of 5.025 Gpc. It was implemented using the HACC simulation and analysis framework on the BG/Q system Mira. The cosmological parameters are chosen to be consistent with the results from the Planck satellite. A range of analysis tools have been run in situ to enable a diverse set of science projects and, at the same time, keep the resulting data amount manageable. Analysis outputs have been generated starting at redshift z ~ 10 to allow for construction of synthetic galaxy catalogs using a semianalytic modeling approach in postprocessing. As part of our in situ analysis pipeline, we employ a new method for tracking halo substructures, introducing the concept of subhalo cores. The production of multiwavelength synthetic sky maps is facilitated by generating particle light cones in situ, also beginning at z ~ 10. We provide an overview of the simulation setup and generated data products; a first set of analysis results is presented. A subset of the data is publicly available.

79 ASTRONOMY AND ASTROPHYSICS↗

DESC DC2 Data Release Note

In preparation for cosmological analyses of the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST), the LSST Dark Energy Science Collaboration (LSST DESC) has created a 300 deg$^2$ simulated survey as part of an effort called Data Challenge 2 (DC2). The DC2 simulated sky survey, in six optical bands with observations following a reference LSST observing cadence, was processed with the LSST Science Pipelines (19.0.0). In this Note, we describe the public data release of the resulting object catalogs for the coadded images of five years of simulated observations along with associated truth catalogs. We include a brief description of the major features of the available data sets. To enable convenient access to the data products, we have developed a web portal connected to Globus data services. We describe how to access the data and provide example Jupyter Notebooks in Python to aid first interactions with the data. We welcome feedback and questions about the data release via a GitHub repository.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The Mira-Titan Universe. III. Emulation of the Halo Mass Function

We construct an emulator for the halo mass function over group and cluster mass scales for a range of cosmologies, including the effects of dynamical dark energy and massive neutrinos. The emulator is based on the recently completed Mira-Titan Universe suite of cosmological N-body simulations. The main set of simulations spans 111 cosmological models with 2.1 Gpc boxes. We extract halo catalogs in the redshift range z = [0.0, 2.0] and for masses M-200c >= 10(13)M(circle dot)/h. The emulator covers an eight-dimensional hypercube spanned by {Omega(m)h(2), Omega(b)h(2), Omega(nu)h(2), sigma(8), h, n(s), w(0), w(a)}; spatial flatness is assumed. We obtain smooth halo mass functions by fitting piecewise second-order polynomials to the halo catalogs and employ Gaussian process regression to construct the emulator while keeping track of the statistical noise in the input halo catalogs and uncertainties in the regression process. For redshifts z less than or similar to 1, the typical emulator precision is better than 2% for 10(13)-10(14)M(circle dot)/h and <10<^> M similar or equal to 101(circle dot)(54)/h. For comparison, fitting functions using the traditional universal form for the halo mass function can be biased at up to 30% at M similar or equal to 10(144)M(circle dot)/h for z = 0. Our emulator is publicly available at https://github.com/SebastianBocquet/MiraTitanHMFemulator.

cosmology: theory↗

The Outer Rim Simulation: A Path to Many-core Supercomputers

Herein we describe the Outer Rim cosmological simulation, one of the largest high-resolution N-body simulations performed to date, aimed at promoting science to be carried out with large-scale structure surveys. The simulation covers a volume of (4.225 Gpc) 3 and evolves more than one trillion particles. It was executed on Mira, a BlueGene/Q system at the Argonne Leadership Computing Facility. We discuss some of the computational challenges posed by a system such as Mira, a many-core supercomputer, and how the simulation code, Hardware/Hybrid Accelerated Cosmology Code, has been designed to overcome these challenges. We have carried out a large range of analyses on the simulation data and we report on the results as well as the data products that have been generated. The full data set generated by the simulation totals more than 5 PB of data, making curation and handling of the data a large challenge in itself. The simulation results have been used to generate synthetic catalogs for large-scale structure surveys, including Dark Energy Spectroscopic Instrument and eBOSS, as well as experiments on the cosmic microwave background. A detailed catalog for the data challenges of the Large Synoptic Survey Telescope Dark Energy Science Collaboration has been created as well. We publicly release some of the Outer Rim halo catalogs, downsampled particle information, and lightcone data.

79 ASTRONOMY AND ASTROPHYSICS↗

The Borg Cube Simulation: Cosmological Hydrodynamics with CRK-SPH

A challenging requirement posed by next-generation observations is a firm theoretical grasp of the impact of baryons on structure formation. Cosmological hydrodynamic simulations modeling gas physics are vital in this regard. A high degree of modeling flexibility exists in this space, making it important to explore a range of methods in order to gauge the accuracy of simulation predictions. We present results from the first cosmological simulation using Conservative Reproducing Kernel Smoothed Particle Hydrodynamics (CRK-SPH). We employ two simulations: one evolved purely under gravity, and the other with nonradiative hydrodynamics. Each contains 2 × 2304 3 cold dark matter plus baryon particles in an 800 h -1 Mpc box. We compare statistics to previous nonradiative simulations including power spectra, mass functions, baryon fractions, and concentration. We find self-similar radial profiles of gas temperature, entropy, and pressure and show that a simple analytic model recovers these results to better than 40% over two orders of magnitude in mass. We quantify the level of nonthermal pressure support in halos and demonstrate that hydrostatic mass estimates are biased low by 24% (10%) for halos of mass 10 15 (10 13 ) h -1 M⊙. Here, we compute angular power spectra for the thermal and kinematic Sunyaev–Zel’dovich effects and find good agreement with the low-ℓ Planck measurements. Finally, artificial scattering between particles of unequal mass is shown to have a large impact on the gravity-only run, and we highlight the importance of better understanding this issue in hydrodynamic applications. This is the first in a simulation campaign using CRK-SPH, with future work including subresolution gas treatments.

79 ASTRONOMY AND ASTROPHYSICS↗

Halo Profiles and the Concentration–Mass Relation for a ΛCDM Universe

Profiles of dark matter-dominated halos at the group and cluster scales play an important role in modern cosmology. Using results from two very large cosmological N-body simulations, which increase the available volume at their mass resolution by roughly two orders of magnitude, we robustly determine the halo concentration-mass (c-M) relation over a wide range of masses, employing multiple methods of concentration measurement. We then characterize individual halo profiles, as well as stacked profiles, relevant for galaxy-galaxy lensing and next-generation cluster surveys; the redshift range covered is 0 ≤ z ≤ 4, with a minimum halo mass of M 200c ~ 2 x 10 11 M ⊙ . Despite the complexity of a proper description of a halo (environmental effects, merger history, nonsphericity, relaxation state), when the mass is scaled by the nonlinear mass scale M * (z), we find that a simple non-power-law form for the c-M/M * relation provides an excellent description of our simulation results across eight decades in M/M * and for 0 ≤ z ≤ 4. Over the mass range covered, the c-M relation has two asymptotic forms: an approximate power law below a mass threshold M/M * ~500-1000, transitioning to a constant value, c 0 ~ 3 at higher masses. The relaxed halo fraction decreases with mass, transitioning to a constant value of ~0.5 above the same mass threshold. We compare Navarro-Frenk-White (NFW) and Einasto fits to stacked profiles in narrow mass bins at different redshifts; as expected, the Einasto profile provides a better description of the simulation results. At cluster scales at low redshift, however, both NFW and Einasto profiles are in very good agreement with the simulation results, consistent with recent weak lensing observations.

79 ASTRONOMY AND ASTROPHYSICS↗

The Mira-Titan Universe. II. Matter Power Spectrum Emulation

We introduce a new cosmic emulator for the matter power spectrum covering eight cosmological parameters. Targeted at optical surveys, the emulator provides accurate predictions out to a wavenumber k ~ 5Mpc -1 and redshift z ≤ 2. Besides covering the standard set of CDM parameters, massive neutrinos and a dynamical dark energy of state are included. The emulator is built on a sample set of 36 cosmological models, carefully chosen to provide accurate predictions over the wide and large parameter space. For each model, we have performed a high-resolution simulation, augmented with sixteen medium-resolution simulations and TimeRG perturbation theory results to provide accurate coverage of a wide k-range; the dataset generated as part of this project is more than 1.2Pbyte. With the current set of simulated models, we achieve an accuracy of approximately 4%. Because the sampling approach used here has established convergence and error-control properties, follow-on results with more than a hundred cosmological models will soon achieve ~1% accuracy. We compare our approach with other prediction schemes that are based on halo model ideas and remapping approaches. The new emulator code is publicly available.

97 MATHEMATICS AND COMPUTING↗

Next-Generation Cosmology Simulations with HACC: Conquering the Baryon Problem

The next generation of cosmological surveys will start taking data in a few years. These surveys will cover the microwave, optical, and X-ray bands. A major challenge for the interpretation of these new observations will be to disentangle astrophysical effects (e.g., galaxy bias and evolution, baryonic feedback processes) from fundamental physics (dark energy, dark matter, neutrino properties, primordial fluctuations).

79 ASTRONOMY AND ASTROPHYSICS↗

High Energy Physics Exascale Requirements Review. An Office of Science review sponsored jointly by Advanced Scientific Computing Research and High Energy Physics, June 10-12, 2015, Bethesda, Maryland

The U.S. Department of Energy (DOE) Office of Science (SC) Offices of High Energy Physics (HEP) and Advanced Scientific Computing Research (ASCR) convened a programmatic Exascale Requirements Review on June 10–12, 2015, in Bethesda, Maryland. This report summarizes the findings, results, and recommendations derived from that meeting. The high-level findings and observations are as follows. Larger, more capable computing and data facilities are needed to support HEP science goals in all three frontiers: Energy, Intensity, and Cosmic. The expected scale of the demand at the 2025 timescale is at least two orders of magnitude — and in some cases greater — than that available currently. The growth rate of data produced by simulations is overwhelming the current ability of both facilities and researchers to store and analyze it. Additional resources and new techniques for data analysis are urgently needed. Data rates and volumes from experimental facilities are also straining the current HEP infrastructure in its ability to store and analyze large and complex data volumes. Appropriately configured leadership-class facilities can play a transformational role in enabling scientific discovery from these datasets. A close integration of high-performance computing (HPC) simulation and data analysis will greatly aid in interpreting the results of HEP experiments. Such an integration will minimize data movement and facilitate interdependent workflows. Long-range planning between HEP and ASCR will be required to meet HEP’s research needs. To best use ASCR HPC resources, the experimental HEP program needs (1) an established, long-term plan for access to ASCR computational and data resources, (2) the ability to map workflows to HPC resources, (3) the ability for ASCR facilities to accommodate workflows run by collaborations potentially comprising thousands of individual members, (4) to transition codes to the next-generation HPC platforms that will be available at ASCR facilities, (5) to build up and train a workforce capable of developing and using simulations and analysis to support HEP scientific research on next-generation systems.

97 MATHEMATICS AND COMPUTING↗

THE MIRA–TITAN UNIVERSE: PRECISION PREDICTIONS FOR DARK ENERGY SURVEYS

Large-scale simulations of cosmic structure formation play an important role in interpreting cosmological observations at high precision. The simulations must cover a parameter range beyond the standard six cosmological parameters and need to be run at high mass and force resolution. A key simulation-based task is the generation of accurate theoretical predictions for observables using a finite number of simulation runs, via the method of emulation. Using a new sampling technique, we explore an eight-dimensional parameter space including massive neutrinos and a variable equation of state of dark energy. We construct trial emulators using two surrogate models (the linear power spectrum and an approximate halo mass function). The new sampling method allows us to build precision emulators from just 26 cosmological models and to systematically increase the emulator accuracy by adding new sets of simulations in a prescribed way. Emulator fidelity can now be continuously improved as new observational data sets become available and higher accuracy is required. Finally, using one ΛCDM cosmology as an example, we study the demands imposed on a simulation campaign to achieve the required statistics and accuracy when building emulators for investigations of dark energy.

79 ASTRONOMY AND ASTROPHYSICS↗

COSMIC EMULATION: FAST PREDICTIONS FOR THE GALAXY POWER SPECTRUM

The halo occupation distribution (HOD) approach has proven to be an effective method for modeling galaxy clustering and bias. In this approach, galaxies of a given type are probabilistically assigned to individual halos in N-body simulations. In this paper, we present a fast emulator for predicting the fully nonlinear galaxy–galaxy auto and galaxy–dark matter cross power spectrum and correlation function over a range of freely specifiable HOD modeling parameters. The emulator is constructed using results from 100 HOD models run on a large ΛCDM N-body simulation, with Gaussian Process interpolation applied to a PCA-based representation of the galaxy power spectrum. The total error is currently ∼1% in the auto correlations and ∼2% in the cross correlations from z = 1 to z = 0, over the considered parameter range. We use the emulator to investigate the accuracy of various analytic prescriptions for the galaxy power spectrum, parametric dependencies in the HOD model, and the behavior of galaxy bias as a function of HOD parameters. Additionally, we obtain fully nonlinear predictions for tangential shear correlations induced by galaxy–galaxy lensing from our galaxy–dark matter cross power spectrum emulator. All emulation products are publicly available at http://www.hep.anl.gov/cosmology/CosmicEmu/emu.html.

79 ASTRONOMY AND ASTROPHYSICS↗