Search NASA⌕ Search

Engineering topics

Messer II, Bronson

Publications and source records attributed to Messer II, Bronson.

Towards Sustainable Post-Exascale Leadership Computing

As computing systems approach the limits of traditional silicon technology, the diminishing returns in performance per watt present a significant barrier to sustaining growth in HPC. From a large-scale scientific supercomputing facility point of view, we propose a multifaceted strategy toward specialized hardware and architectures that are optimized for energy efficiency in specific applications. We also emphasize the need for integrating energy-aware practices across all levels of HPC, from system design and software development to operational policies. We discuss strategic opportunities such as the adoption of application-specific accelerators, the development of energy-efficient algorithms, and the implementation of data-driven operational analytics. Our goal is to develop a comprehensive roadmap ensuring that future leadership systems at OLCF can meet scientific demands while operating within stringent energy budgets, thereby supporting sustainable computing growth.

Shin, Woong↗

OLCF’s Advanced Computing Ecosystem (ACE): FY24 Efforts for the DOE Integrated Research Infrastructure (IRI) Program

This report highlights significant strides made by Oak Ridge National Laboratory’s Oak Ridge Leadership Computing Facility (OLCF) in advancing computational research and infrastructure. Through the Advanced Computing Ecosystem (ACE) strategic initiative, OLCF has been successfully integrated with Doe’s Integrated Research Infrastructure (IRI) program, establishing itself as a critical framework for enhancing scientific computing capabilities across various domains. This report outlines the activities, accomplishments, and future directions of ACE, emphasizing its role in developing cutting-edge technologies, supporting science pilots, and fostering collaborations that drive scientific innovation.

97 MATHEMATICS AND COMPUTING↗

Experiences readying applications for Exascale

The advent of Exascale computing invites an assessment of existing best practices for developing application readiness on the world's largest supercomputers. This work details observations from the last four years in preparing scientific applications to run on the Oak Ridge Leadership Computing Facility's (OLCF) Frontier system. This paper addresses a range of topics in software including programmability, tuning, and portability considerations that are key to moving applications from existing systems to future installations. A set of representative workloads provides case studies for general system and software testing. We evaluate the use of early access systems for development across several generations of hardware. Finally, we discuss how best practices were identified and disseminated to the community through a wide range of activities including user-guides and trainings. We conclude with recommendations for ensuring application readiness on future leadership computing systems.

Gottiparthi, Kalyan↗

Ready for the Frontier: Preparing Applications for the World’s First Exascale System

Frontier, a supercomputer at the Oak Ridge Leadership Computing Facility (OLCF), debuted atop the Top500 list of the world’s most powerful supercomputers in June 2022 as the very first computer to produce exascale performance. Making sure scientific applications are optimized on this architecture is the critical link necessary to translate the newly available computational power into scientific insight and solutions. To that goal, the OLCF developed the Center for Accelerated Application Readiness (CAAR) program to ensure that a suite of highly optimized applications is ready for scientific runs at the onset of production operations for Frontier. This paper describes our experience in porting and optimizing such suite of applications in the OLCF’s CAAR program.

Budiardja, Reuben↗

High Performance Computing Facility Operational Assessment 2022: Oak Ridge Leadership Computing Facility

The Oak Ridge Leadership Computing Facility (OLCF) was established to accelerate scientific discovery by providing world-leading computational performance and advanced data infrastructure. As a US Department of Energy (DOE) Office of Science user facility, the OLCF has managed the successful deployment and operation of a succession of leadership-class resources dedicated to open science. In addition to these resources, the OLCF staff continually strive to develop innovative processes and technologies, improve security, and empower users through allocation management and comprehensive user support and training. These efforts support the advancement of science by the OLCF users and benefit high-performance computing (HPC) facilities around the world. In calendar year (CY) 2022, the OLCF supported 1,681 users and 570 projects and exceeded all targets for user satisfaction. The facility received an average satisfaction score of 4.6 out of 5 on the annual user survey, and 96% of respondents reported a high satisfaction rate with the OLCF overall. Of the 3,212 user tickets submitted in CY 2022, OLCF staff resolved 97% within 3 business days. The facility also introduced several new services for users this year, including weekly virtual office hours with subject matter experts from ORNL and vendor partners; new views in MyOLCF that allow users to analyze allocation and compute usage for a project; the ability to build and run containers on Summit; and improved data visualization support and training resources.

97 MATHEMATICS AND COMPUTING↗

Application Experiences on a GPU-Accelerated Arm-based HPC Testbed

This paper assesses and reports the experience of ten teams working to port, validate, and benchmark several High Performance Computing applications on a novel GPU-accelerated Arm testbed system. The testbed consists of eight NVIDIA Arm HPC Developer Kit systems, each one equipped with a server-class Arm CPU from Ampere Computing and two data center GPUs from NVIDIA Corp. The systems are connected together using InfiniBand interconnect. The selected applications and mini-apps are written using several programming languages and use multiple accelerator-based programming models for GPUs such as CUDA, OpenACC, and OpenMP offloading. Working on application porting requires a robust and easy-to-access programming environment, including a variety of compilers and optimized scientific libraries. The goal of this work is to evaluate platform readiness and assess the effort required from developers to deploy well-established scientific workloads on current and future generation Arm-based GPU-accelerated HPC systems. The reported case studies demonstrate that the current level of maturity and diversity of software and tools is already adequate for large-scale production deployments.

Elwasif, Wael↗

Flash-X: A multiphysics simulation software instrument

Flash-X is a highly composable multiphysics software system that can be used to simulate physical phenomena in several scientific domains. It derives some of its solvers from FLASH, which was first released in 2000. Flash-X has a new framework that relies on abstractions and asynchronous communications for performance portability across a range of increasingly heterogeneous hardware platforms. Flash-X is meant primarily for solving Eulerian formulations of applications with compressible and/or incompressible reactive flows. It also has a built-in, versatile Lagrangian framework that can be used in many different ways, including implementing tracers, particle-in-cell simulations, and immersed boundary methods.

97 MATHEMATICS AND COMPUTING↗

Learning to Scale the Summit: AI for Science on a Leadership Supercomputer

The Summit system at Oak Ridge National Lab-oratory (ORNL) has been the world's top AI for science su-percomputer for several years, ranked world's fastest computer at its 2018 launch and currently top system in the US and #2 on the TOP5OO list. Summit's purposeful design to handle both conventional modeling and simulation science and emerging AI workloads has made it a leading destination for AI-powered computational science. We report here on AI for science usage on Summit near the midpoint of its lifespan. We review AI usage across the many science projects that have used Summit. We then examine in detail a set of applications scaling AI to full system as well as projects implementing AI-coordinated science discovery workflows on Summit. Finally, we offer some observations regarding the future of advancing scientific knowledge and understanding via AI, especially in the context of leadership-class scientific computing.

Joubert, Wayne↗

Utilizing IBM Spectrum LSF Simulator to Understand the Impacts of Adding AI Workloads to Capability Supercomputing

Machine Learning and Artificial Intelligence has been identified as an emerging priority science area within the Department of Energy. Large scale accelerator based supercomputers like Summit, while traditionally employed for modeling and simulation, provide architectures that are suitable for accelerating the ML/AI workloads at scale. With the release of Summit in 2018, there was an increase in the number of ML/AI based projects seeking time on the machine. It quickly became apparent that the allocations and job runtimes for this workload deviated from traditional large scale modeling and simulation. Accommodating this new workload requires understanding the impacts to traditional large scale modeling and simulation.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

On the character of turbulent-like flows in self-consistent models of core-collapse supernovae

Neutrino-driven convection plays a crucial role in the development of core-collapse supernova (CCSN) explosions. However, the complex mechanism that triggers the shock revival and the subsequent explosion has remained inscrutable for many decades. Multidimensional simulations suggest that the growth of fluid instabilities, especially development of convection, will determine the morphology of the explosion. We have performed 3D simulations using spherical-polar coordinates covering a reduced angular extent (90° computational domain), and with angular resolutions of 2°, 1°, 1/2°, and 1/4°, to study the development of turbulent-like flows in core-collapse supernova explosions on a time scale of order 100 ms. We have employed the multi-physics Chimera code that includes detailed nuclear physics and spectral neutrino transport. Coarse resolution models do not develop an inertial range, presumably due to the bottleneck effect, such that the energy is prevented from cascading down to small scales and tends to accumulate at large scales. High-resolution models instead, start to recover the k −5/3 scaling of Kolmogorov's theory. Stochasticity and few simulation samples limit our ability to predict the development of explosions. Over the simulated time period, our models show no clear trend in improving (or diminishing) conditions for explosion as the angular resolution is increased. However, we find that disordered flow provides an effective pressure, as characterized by the Reynolds stress, behind the shock. This contribution, ~40%–50% of the thermal pressure, aids shock revival and thus the development of the explosion. Finally, we show that the kinetic energy power spectrum of reduced angular extent and full 4π models are consistent, thus indicating that a 90° computational domain is an adequate configuration to study the character of turbulence in CCSNe.

Casanova Bustamante, Jordi↗

US Department of Energy, Office of Science High Performance Computing Facility Operational Assessment 2019 Oak Ridge Leadership Computing Facility

Oak Ridge National Laboratory's (ORNL's) Leadership Computing Facility (OLCF) continues to surpass its operational target goals: supporting users; delivering fast, reliable computational ecosystems; creating innovative solutions for high performance computing (HPC) needs; and managing risks, safety, and security associated with operating some of the most powerful computers in the world. The results can be seen in the cutting-edge science conducted by users and the praise from the research community. Calendar year (CY) 2019 was a big year as OLCF staff ran five world-class resources (the leadershipclass computers Titan and Summit, the large analysis cluster called Eos, and the massive parallel filesystems called Atlas and Alpine)) and also began power and cooling upgrades for a 2021 exascale system called Frontier. While continuing exceptional operation of Titan, Eos, and Rhea, the OLCF released the Summit supercomputer for production on January 1, 2019. Summit debuted as the most capable and efficient system in its class and has been recognized as the most powerful system in the world for its performance on both the high performance linpack (HPL) and conjugate gradient (HPCG) benchmark applications since June 2018 according to TOP500. Summit represents the culmination of a multiyear effort between the OLCF, IBM, NVIDIA, and Mellanox to deliver a system that is unmatched for modeling, simulation, data analysis, and learning. To hit the ground running with science-ready applications on day one, application teams worked closely with the OLCF through the Center for Accelerated Application Readiness (CAAR) program for years in advance of the Summit deployment. CY 2019 was filled with outstanding results and accomplishments: a very high rating from users on overall satisfaction for the sixth year in a row; a tremendous amount of core-hours delivered to researchers from two leadership-class systems; and success in delivering on the allocation split of roughly 60%, 30%, and 10% of core-hours offered for the Innovative and Novel Computational Impact on Theory and Experiment (INCITE), Advanced Scientific Computing Research Leadership Computing Challenge (ALCC), and Director's Discretionary (DD) programs, respectively (see Operational Performance section). These accomplishments, coupled with the high utilization rates (overall and capability usage), represent the fulfillment of the promise of both leadership-class machines: efficient facilitation of leadership-class computational applications. Table ES.1 presents a summary of the 2019 OLCF metric targets and the associated results. More information can be found in the Operational Performance section for each OLCF resource. The scientific accomplishments of OLCF users are a strong indication of long-term operational success, with publications this year in such notable journals and publications as Nature, Nature Physics, Nature Plants, Physical Review X, Journal of the American Physical Society, Cell, Nano Letters, and Trends in Biotechnology. Crucial domain-specific discoveries facilitated by resources at the OLCF are described in the High Performance Computing Facility Operational Assessment 2019 Oak Ridge Leadership Computing Facility (OAR) Strategic Results section. For example, researchers used Summit to pinpoint and understand the production of proteins from genetic information, including mutations and the functional expression of disease (Section 8.2).

97 MATHEMATICS AND COMPUTING↗

Learning About Nucleosynthesis from Multi-dimensional Simulations of Core-Collapse Supernovae

For more than two decades, we have understood that the development of a successful core-collapse supernova is inextricably linked to neutrino heating and three dimensional fluid flows, with large scale hydrodynamic instabilities allowing successful explosions that spherical symmetry would prevent. Unfortunately, our understanding of the nucleosynthesis that occurs in these supernovae, and therefore the impact of supernovae on galactic chemical evolution, has generally ignored much that we have learned about the central engine of these supernovae over the past two decades. Now, with two and three dimensional simulations of core-collapse supernovae run to sufficient duration, we are learning how the multi-dimensional, neutrino-driven character of the explosions directly impacts the nucleosynthesis and other observables of core-collapse supernovae. Here we focus on lessons from multi-dimensional models which implement realistic nuclear reaction networks.

Hix, W Raphael↗

thornado-transport: IMEX schemes for two-moment neutrino transport respecting Fermi-Dirac statistics

We develop implicit-explicit (IMEX) schemes for neutrino transport in a background material in the context of a two-moment model that evolves the angular moments of a neutrino phase-space distribution function. Considering the upper and lower bounds that are introduced by Pauli's exclusion principle on the moments, an algebraic moment closure based on Fermi-Dirac statistics and a convex-invariant time integrator both are demanded. A finite-volume/first-order discontinuous Galerkin(DG) method is used to illustrate how an algebraic moment closure based on Fermi-Dirac statistics is needed to satisfy the bounds. Several algebraic closures are compared with these bounds in mind, and the Cernohorsky and Bludman closure, which satisfies the bounds, is chosen for our IMEX schemes. For the convex-invariant time integrator, two IMEX schemes named PD-ARS have been proposed. PD-ARS denotes a convex-invariant IMEX Runge-Kutta scheme that is high-order accurate in the streaming limit, and works well in the diffusion limit. Our two PD-ARS schemes use second-and third-order, explicit, strong-stability-preserving Runge-Kutta methods as their explicit part, respectively, and therefore are second-and third-order accurate in the streaming limit, respectively. The accuracy and convex-invariance of our PD-ARS schemes are demonstrated in the numerical tests with a third-order DG method for spatial discretization and a simple Lax-Friedrichs flux. The method has been implemented in our high-order neutrino-radiation hydrodynamics (thornado) toolkit. We show preliminary results employing tabulated neutrino opacities.

Chu, Ran↗

Multi-dimensional simulations of core-collapse supernova explosions with CHIMERA

Unraveling the core-collapse supernova (CCSN) mechanism is a problem that remains essentially unsolved despite more than four decades of effort. Spherically symmetric models with otherwise high physical fidelity generally fail to produce explosions, and it is widely accepted that CCSNe are inherently multi-dimensional. Progress in realistic modeling has occurred recently through the availability of petascale platforms and the increasing sophistication of supernova codes. We will discuss our most recent work on understanding neutrino-driven CCSN explosions employing multi-dimensional neutrino-radiation hydrodynamics simulations with the Chimera code. We discuss the inputs and resulting outputs from these simulations, the role of neutrino radiation transport, and the importance of multi-dimensional fluid flows in shaping the explosions. We also highlight the production of 48Ca in long-running Chimera simulations.

Messer II, Bronson↗

Towards a Discontinuous Galerkin Method for the Multi-Group Two-Moment Model of Neutrino Transport

We present progress on the development of a Discontinuous Galerkin (DG) method for solving the non-relativistic, multi-group two-moment model for neutrino transport. The method - ultimately targeted for simulation of neutrino transport in core-collapse supernovae - includes the use of curvilinear coordinates and tabulated microphysics (i.e., nuclear equation of state and weak interaction rates) to model neutrino-matter coupling. In this report, we provide algorithm details and present results from an extensive set of benchmark problems in one spatial dimension. The algorithms have been implemented in the Toolkit for High-Order Neutrino-Radiation Hydrodynamics ( thornado ), and Fortran code used to generate the results has been made available to the public through GitHub.

74 ATOMIC AND MOLECULAR PHYSICS↗

Advancements in modeling self-consistent core collapse supernovae with CHIMERA

We discuss advancements in modeling core-collapse supernovae with our code CHIMERA. We describe the status and details of our tracer particle method and its uses for post-processing nucleosynthesis and as a tool for broad core-collapse supernova (CCSN) model analyses. We also introduce our progress towards expanding a nuclear reaction network beyond the alphanetwork for the purpose of accurate in-situ nucleosynthesis, not only for core-collapse supernovae (CCSNe) in general, but for a special sub-class of supernovae called electron capture supernovae (ECSNe), which stem from progenitors stars between 8 and 10 M⊙. By using an advanced nuclear reaction network, our 2D and 3D code will allow for unparalleled studies of CCSNe and ECSNe ejecta.

79 ASTRONOMY AND ASTROPHYSICS↗