NASA GPU Hackathon 2022 Summary
Explore the source record for details and available documents.
Haoqiang Jin↗
Engineering topics
Gabriele Jost
Publications and source records attributed to Gabriele Jost.
NASA GPU Hackathon Yields Significant Code Improvements
The NASA GPU Hackathon 2020 brought together application developers and computer experts to help get important NASA applications running effectively on graphics processing unit (GPU) nodes. Nine teams of application developers participated in this virtual event, a major impetus for teams to modernize codes of interest for NASA missions to CPU nodes containing GPU accelerators, with a focus on hands-on problem solving. The photo in Figure1 shows 30 of the more than50 participants. The HECC project and NVIDIA jointly organized the event, and HECC provided five Pleiades nodes each with 4 V100 GPUs for teams to use. The virtual event, which took place over four days from September 28–October 7, 2020, used Microsoft Teams and Slack as collaboration tools. Each team consisted of three to six members from NASA Centers and supporting organizations. The teams were paired with one to two mentors from industry, government, and academia. The experience levels of the teams ranged from being GPU novices to advanced CUDA programming experts. OpenACC and the emerging Kokkos API were used in addition to CUDA for GPU programming. During the event, which focused on accelerating AeroSciences and CFD applications, most teams achieved considerable performance improvements on both GPUs and CPUs. For example, a team with no GPU experience completed a first port of a time-critical loop to a GPU. Another team of expert CUDA programmers were able to restructure their algorithm, yielding a factor of five speed-up. And another team sped up some of their CUDA kernels by a factor of 20, which directly translated into their production code. This article highlights some of the many successes resulting from the event.
HECC↗
Enabling Execution of a Legacy CFD Mini Application on Accelerators Using OpenMP
We describe the process and outcome of our efforts to port a legacy Fortran benchmark code to heterogeneous GPU-accelerated computing architectures using OpenMP. The benchmark code is one of the multi-zone NAS Parallel Benchmarks (NPB-MZ) called SP-MZ. This “mini-app” mimics the computation and data movement that is found in popular legacy and modern implicit computational fluid dynamics (CFD)solvers. Our objective was to examine how efficiently legacy Fortran codes can be ported to accelerators by leveraging OpenMP directives. We describe the development and optimization process and demonstrate the performance impact of various code modifications. We show select profiling results from the Nvidia nvvp profiler to help others diagnose and overcome performance issues in their own applications. We present results for two compute systems endowed with Nvidia V100 accelerators.
Ioannis Nompelis↗
Adding GPU Support to the Markov Chain Monte Carlo Code Catmip
In geophysics, we are confronted with many under-determined inverse problems. For example, all of our observations of earthquakes are made at the Earth’s surface. So, when we try to infer how slip during an earthquake evolves in space and time, we find that there are many potential slip histories that are consistent with our limited observations and our understanding of earthquake physics. One way to approach these problems is with Bayesian analysis which allows us to infer the ensemble of all potential slip models that satisfy the observations and our prior knowledge of earthquake physics. In Bayesian analysis, our prior knowledge is known as the prior probability density function or prior PDF, the fit to the data is known as the data likelihood, and the target PDF that satisfies both the prior PDF and data likelihood is known as the posterior PDF. However, simulating the posterior PDF typically requires using Markov Chain Monte Carlo (MCMC) to draw tens of billions of random realizations of earthquake slip models, which may not be computationally feasible. To make this and similar geophysical inversions computationally tractable, we developed the Cascading Adaptive Transitional Metropolis In Parallel (CATMIP) algorithm. CATMIP is an efficient parallel Markov Chain Monte Carlo (MCMC) sampler that is used for model fitting and uncertainty quantification in geophysics. Example use cases are earthquake rupture modeling, determining mineral composition on Mars, reconstructing the history of ocean salinity, and historical earthquake relocation. CATMIP employs many parallel instances of the Metropolis algorithm for sampling in a transitioning framework. Transitioning is a process in which a set of random samples at equilibrium with a known probability density function (PDF) are used as seeds for the Markov chains to sample successive target PDFs that incrementally move the distribution from the starting seeds to the final desired PDF that describes the relative plausibility of potential values for the model parameters. The algorithm is implemented as a Master-Worker model employing MPI for communication. The worker processes are loosely coupled with global parameters periodically optimized by the master process. This provides a very high amount of parallelism with little communication between updates. During the presentation we will discuss the history of the algorithm and elaborate the earthquake rupture modeling use case for the CATMIP package. Our first step toward GPU optimization was to optimize the code for the CPU. CPU profiling revealed that most of the compute time is spent in calls to level 2 BLAS routines and calls to GSL random number generators. We revised the algorithm to employ level 3 BLAS routines instead. In our presentation we will describe how this was accomplished. Adding GPU support to CATMIP consisted mostly of replacing the calls to GSL with calls to GPU vendor-provided library routines. A small number of loops were directly implemented in CUDA. In the presentation will provide implementation details. Finally, we will discuss methods for profiling and opportunities for further optimizing GPU execution. By creating a code with the flexibility to run on either a CPU or GPU architecture, CATMIP can be used on systems ranging from large CPU-based HPC environments to single servers with GPU acceleration and everything in between.
HECC↗
A GPU-based Approach for Turbomachinery Application
Accurate modeling of coolant airflows, which form protective films over turbine blades, is essential for designing fuel-efficient and environmentally sustainable gas turbine engines. Excessive coolant reduces thermal efficiency, while insufficient coolant leads to blade overheating and causes damage. Therefore, precise prediction of flow field interactions with cooling air is critical for optimizing turbine performance. This study numerically investigates the cooling effectiveness of purge and film cooling flows within a high-pressure turbine (HPT) rotor using Large Eddy Simulation (LES). The study utilizes NASA Glenn Research Center’s Glenn-HT solver. The simulation models ethe conditions of the Penn State University START rotating rig. A high-fidelity structured mesh comprising up to 800 million cells is employed to resolve high-Reynolds number flow (Re ≈ 350,000) and to capture intricate secondary flow structures, including tip leakage and purge-induced vortices. Film cooling effectiveness computations are highly sensitive to boundary conditions at the cooling holes and to grid resolution. Even with well-resolved grids and included plena, strong mixing challenges traditional eddy viscosity models. To address this, a simplified configuration is simulated: a truncated row of shaped holes on the suction side near the leading edge and a row on the pressure side, both fed from internal plena while the purge slot and tip clearance are also modeled. Two isothermal LES cases are conducted at two distinct wall temperatures, which yield the adiabatic wall temperature and the heat transfer coefficient. The definition and means of computation of the effectiveness is discussed in this paper. The simulations reveal detailed three-dimensional unsteady flow features, including coherent vortical structures and secondary flows originating from the purge cavity. Film cooling effectiveness and Nusselt number distributions are presented for both the blade surface and tip, highlighting regions of elevated heat transfer and complex thermal behavior. These findings underscore the importance of high-resolution LES and realistic boundary conditions in capturing the dynamics of purge and film cooling, offering valuable insights for improving turbine blade design and thermal management strategies.
Gas Turbine↗