Portable Acceleration of Materials Modeling Software: CASTEP, GPUs, and OpenACC
Not provided.
Computer Science↗
SEARCH · Search NASA
Results for “OpenACC”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
OpenACC Profiling Support for Clang and LLVM using Clacc and TAU
Since its launch in 2010, OpenACC has evolved into one of the most widely used portable programming models for accelerators on HPC systems today. Clacc is a project funded by the US Exascale Computing Project (ECP) to bring OpenACC support for C and C++ to the popular Clang and LLVM compiler infrastructure. In this paper, we describe Clacc's support for the OpenACC Profiling Interface, a critical component of the OpenACC specification that standardizes an interface that profiling tools and libraries can depend upon across OpenACC implementations. As part of Clacc's general strategy to build OpenACC support upon OpenMP, we describe how Clacc builds OpenACC Profiling Interface support upon an extended version of OMPT. We then describe how a major profiling and tracing toolkit within ECP, the TAU Performance System, takes advantage of this support. We also describe TAU's selective instrumentation support for OpenACC. Finally, using Clacc and TAU, we present example visualizations for several SPEC ACCEL OpenACC benchmarks running on an IBM AC922 node, and we show that the associated performance overhead is negligible.
Coti, Camille↗
Clacc: OpenACC for C/C++ in Clang
The Clacc project has developed OpenACC compiler, runtime, and profiling interface support for C/C++ by extending Clang and LLVM. A key Clacc design feature is that it translates OpenACC to OpenMP to leverage the OpenMP offloading support that is actively being developed for Clang and LLVM. A benefit of this design is support for two compilation modes: traditional compilation mode produces a binary, and source-to-source mode produces OpenMP source. Clacc has been deployed on Oak Ridge National Laboratory’s (ORNL’s) Frontier, on which Clacc is the only OpenACC implementation for C/C++. Clacc supports x86_64, POWER9, AMD GPUs, and NVIDIA GPUs. Clacc’s OpenACC profiling interface support has been integrated with TAU, which is also deployed on Frontier. While Clacc has always supported C as a base language, Clacc also has increasing C++ support, including support for Kokkos’s OpenACC back end. Clacc itself is hosted publicly on GitHub. In this paper, we describe Clacc’s design and mapping from OpenACC directives to OpenMP. We also present a performance evaluation on ORNL’s Frontier (AMD MI250x GPU offload) and Argonne National Laboratory’s (ANL’s) Polaris (NVIDIA A100 GPU offload) for various SPEC ACCEL and Kokkos OpenACC back end benchmarks.
97 MATHEMATICS AND COMPUTING↗
CCAMP: OpenMP and OpenACC Interoperable Framework
Heterogeneous systems have become a staple of the HPC environment. Several directive-based solutions, such as OpenMP and OpenACC, have been developed to alleviate the challenges of programming heterogeneous systems, and these standards strive to provide a single portable programming solution across heterogeneous environments. However, in many ways this goal has yet to be realized due to device-specific implementations and different levels of language support across compilers. In this framework we aim to analyze and address the different levels of optimization and compatibility between OpenACC and OpenMP programs and device compilers. We introduce the CCAMP framework, built on the OpenARC compiler, which implements language translation between OpenACC and OpenMP, with the goal of exploiting the maturity of different device-specific compilers to maximize performance for a given architecture. We show that CCAMP allows us to generate code for a specific device-compiler combination given a device-agnostic OpenMP or OpenACC program, allowing compilation and execution of programs with specific directives on otherwise incompatible devices. CCAMP also provides a starting point for a more advanced interoperable framework that can effectively provide directive translation and device, compiler, and application specific optimizations.
Lambert, Jacob↗
Experiences in porting mini-applications to OpenACC and OpenMP on heterogeneous systems
This article studies mini-applications—Minisweep, GenASiS , GPP, and FF—that use computational methods commonly encountered in HPC. We have ported these applications to develop OpenACC and OpenMP versions, and evaluated their performance on Titan (Cray XK7 with K20x GPUs), Cori (Cray XC40 with Intel KNL), Summit (IBM AC922 with Volta GPUs), and Cori-GPU (Cray CS-Storm 500NX with Intel Skylake and Volta GPUs). Our goals are for these new ports to be useful to both application and compiler developers, to document and describe the lessons learned and the methodology to create optimized OpenMP and OpenACC versions, and to provide a description of possible migration paths between the two specifications. Cases where specific directives or code patterns result in improved performance for a given architecture are highlighted. Here, we also include discussions of the functionality and maturity of the latest compilers available on the above platforms with respect to OpenACC or OpenMP implementations.
97 MATHEMATICS AND COMPUTING↗
Developing an ELM Ecosystem Dynamics Model on GPU with OpenACC
Porting a complex scientific code, such as the E3SM land model (ELM), onto a new computing architecture is challenging. The paper presents design strategies and technical approaches to develop an ELM ecosystem dynamics model with compiler directives (OpenACC) on NVIDIA GPUs. The code has been refactored with advanced OpenACC features (such as deepcopy and routine directives) to reduce memory consumption and to increase the levels of parallelism through parallel loop reconstruction and new data structures. As a result, the optimized parallel implementation achieved more than a 140-time speedup (50 ms vs 7600 ms), compared to a naive implementation that uses OpenACC routine directive and parallelizes the code across existing loops on a single NVIDIA V100. On a fully loaded computing node with 44 CPUs and 6 GPUs, the code achieved over a 3.0-times speedup, compared to the original code on the CPU. Furthermore, the memory footprint of the optimized parallel implementation is 300 MB, which is around 15% of the 2.15 GB of memory consumed by a naive implementation. This study is the first effort to develop the ELM component on GPUs efficiently to support ultra-high-resolution land simulations at continental scales.
Schwartz, Peter↗
Static Graphs for Coding Productivity in OpenACC
The main contribution of this work is to increase the coding productivity for GPU programming by using the concept of Static Graphs. To do so, we have combined the new CUDA Graph API with the OpenACC programming model. We use as test cases a well-known and widely used problems in HPC and AI: the Particle Swarm Optimization. We complement the OpenACC functionality with the use of CUDA Graph, achieving accelerations of more than one order of magnitude, and a performance very close to a reference and optimized CUDA code. Finally, we propose a new specification to incorporate the concept of Static Graphs into the OpenACC specification.
Toledo, Leonel↗
Analysis of Validating and Verifying OpenACC Compilers 3.0 and Above
OpenACC is a high-level directive-based parallel programming model that can manage the sophistication of heterogeneity in architectures and abstract it from the users. The portability of the model across CPUs and accelerators has gained the model a wide variety of users. This means it is also crucial to analyze the reliability of the compilers’ implementations. To address this challenge, the OpenACC Validation and Verification team has proposed a validation testsuite to verify the OpenACC implementations across various compilers with an infrastructure for a more streamlined execution. This paper will cover the following aspects: (a) the new developments since the last publication on the testsuite, (b) outline the use of the infrastructure, (c) discuss tests that highlight our workflow process, (d) analyze the results from executing the testsuite on various systems, and (e) outline future developments.
Jarmusch, Aaron↗
KokkACC: Enhancing Kokkos with OpenACC
Template metaprogramming is gaining popularity as a high-level solution for achieving performance portability on heterogeneous computing resources. Kokkos is a representative approach that offers programmers high-level abstractions for generic programming while most of the device-specific code generation and optimizations are delegated to the compiler through template specializations. For this, Kokkos provides a set of device-specific code specializations in multiple back ends, such as CUDA and HIP. Unlike CUDA or HIP, OpenACC is a high-level and directive-based programming model. This descriptive model allows developers to insert hints (pragmas) into their code that help the compiler to parallelize the code. The compiler is responsible for the transformation of the code, which is completely transparent to the programmer. This paper presents an OpenACC back end for Kokkos: KokkACC. As an alternative to Kokkos’s existing device-specific back ends, KokkACC is a multi-architecture back end providing a high-productivity programming environment enabled by OpenACC’s high-level and descriptive programming model. Moreover, we have observed competitive performance; in some cases, KokkACC is faster (up to 9×) than NVIDIA’s CUDA back end and much faster than OpenMP’s GPU offloading back end. This work also includes implementation details and a detailed performance study conducted with a set of mini-benchmarks (AXPY and DOT product) and three mini-apps (LULESH, miniFE and SNAP, a LAMMPS proxy mini-app).
Valero Lara, Pedro↗
OpenACC unified programming environment for GPU and FPGA multi-hybrid acceleration
Attached accelerators have been frequently used in recent High Per- formance Computing (HPC) systems because of their high performance/power ratio. In particular, the Graphics Processing Unit (GPU) is the most popu- lar accelerator owing to its high peak FLOPS performance and high memory bandwidth supported by HBM2, etc. However, the performance of GPU depends highly on a large degree of SIMD parallelism and has difficulty sustaining a high performance on programs with frequent branch operations or a partially low degree of parallelism.By contrast, a Field Programmable Gate Array (FPGA) has received attention as a different type of accelerator than GPU as a fully reconfigurable processor fitting the target applications. The high performance of FPGA is mainly provided by a pipelined operation and optimized circuit suitable for any operation even with frequent conditional branches. We have been focusing on the flexibility of FPGA to compensate for the weakness of GPU. We believe that the coupling of GPU with FPGA can result in one of the most powerful accelerating platforms available.However, the program coding of GPU and FPGA coupling can be quite difficult for application users. Traditionally, CUDA by NVIDIA has been the most popular programming language with the largest share of GPUs used in HPC, whereas a hardware description language such as Verilog HDL has been used in FPGA programming. OpenCL coding has recently become available even on high-end FPGAs. Moreover, several recent studies have also enabled the OpenACC coding for use in FPGA. In this study, we provide a unified programming system based on OpenACC for a platform equipped with both GPU and FPGA aiming at the next-generation accelerated supercomputer framework. Our programming environment is called Multi-Hybrid OpenACC Translator (MHOAT), and in this paper, we describe the basic concept and prototype system of MHOAT based on an evaluation on the amount of coding required and the performance of a hybrid multi-device accelerated system.
Tsunashima, Ryuta↗
OpenACC Unified Programming Environment for Multi-hybrid Acceleration with GPU and FPGA
Accelerated computing in HPC such as with GPU, plays a central role in HPC nowadays. However, in some complicated applications with partially different performance behavior is hard to solve with a single type of accelerator where GPU is not the perfect solution in these cases. We are developing a framework and transpiler allowing the users to program the codes with a single notation of OpenACC to be compiled for multi-hybrid accelerators, named MHOAT (Multi-Hybrid OpenACC Translator) for HPC applications. MHOAT parses the original code with directives to identify the target accelerating devices, currently supporting NVIDIA GPU and Intel FPGA, dispatching these specific partial codes to background compilers such as NVIDIA HPC SDK for GPU and OpenARC research compiler for FPGA, then assembles binaries for the final object with FPGA bitstream file. In this paper, we present the concept, design, implementation, and performance evaluation of a practical astrophysics simulation code where we successfully enhanced the performance up to 10 times faster than the GPU-only solution.
Boku, Taisuke↗
CCAMP: An Integrated Translation and Optimization Framework for OpenACC and OpenMP
Heterogeneous computing and exploration into specialized accelerators are inevitable in current and future supercomputers. Although this diversity of devices is promising for performance, the array of architectures presents programming challenges. High-level programming strategies have emerged to face these challenges, such as the OpenMP offloading model and OpenACC. The varying levels of support for these standards, however, within vendor-specific and open-source tools, as well as the lack of performance portability across devices, have prevented the standards from achieving their goals. To address these shortcomings, we present CCAMP, an OpenMP and OpenACC interoperable framework. CCAMP provides two primary facilities: language translation between the two standards and device-specific directive optimization within each standard. We show that by using the CCAMP framework, programmers can easily transplant non-portable code into new ecosystems for new architectures. Additionally, by using CCAMP device-specific directive optimizations, users can achieve optimized performance across architectures using a single source code.
Lambert, Jacob↗
OpenACC offloading of the MFC compressible multiphase flow solver on AMD and NVIDIA GPUs
GPUs are the heart of the latest generations of supercomputers. We efficiently accelerate a compressible multiphase flow solver via OpenACC on NVIDIA and AMD Instinct GPUs. Optimization is accomplished by specifying the directive clauses gang vector and collapse. Further speedups of six and ten times are achieved by packing user-defined types into coalesced multidimensional arrays and manual inlining via metaprogramming. Additional optimizations yield seven-times speedup of array packing and thirty-times speedup of select kernels on Frontier. Weak scaling efficiencies of 97% and 95% are observed when scaling to 50% of Summit and 87% of Frontier. Strong scaling efficiencies of 84% and 81% are observed when increasing the device count by a factor of 8 and 16 on V100 and MI250X hardware. The strong scaling efficiency of AMD’s MI250X increases to 92% when increasing the device count by a factor of 16 when GPU-aware MPI is used for communication.
Wilfong, Benjamin↗
Optimization and Portability of a Fusion OpenACC-based FORTRAN HPC Code from NVIDIA to AMD GPUs
NVIDIA has been the main provider of GPU hardware in HPC systems for over a decade. Most applications that benefit from GPUs have thus been developed and optimized for the NVIDIA software stack. Recent exascale HPC systems are, however, introducing GPUs from other vendors, e.g. with the AMD GPU-based OLCF Frontier system just becoming available. AMD GPUs cannot be directly accessed using the NVIDIA software stack, and require a porting effort by the application developers. This paper provides an overview of our experience porting and optimizing the CGYRO code, a widely-used fusion simulation tool based on FORTRAN with OpenACC-based GPU acceleration. While the porting from the NVIDIA compilers was relatively straightforward using the CRAY compilers on the AMD systems, the performance optimization required more fine-tuning. In the optimization effort, we uncovered code sections that had performed well on NVIDIA GPUs, but were unexpectedly slow on AMD GPUs. After AMD-targeted code optimizations, performance on AMD GPUs has increased to meet our expectations. Modest speed improvements were also seen on NVIDIA GPUs, which was an unexpected benefit of this exercise.
Sfiligoi, Igor↗
sKokkos: Enabling Kokkos with Transparent Device Selection on Heterogeneous Systems using OpenACC
This paper presents a new feature to enable Kokkos with transparent device selection. For application developers, it is not easy toidentify which device is the most appropriate to use in a heterogeneous system, since this depends on the characteristics of both the application and the hardware. In Kokkos, a backend is associated with one specific programming model/hardware. Programmers decide which backend to use at compilation time. This new feature implemented on the OpenACC backend eliminates the burden of deciding which device to use, providing a highly productive programming solution for Kokkos applications. This work includes implementation details and a performance study conducted with a set of mini-benchmarks (i.e., AXPY and dot product), kernels (Lattice-Bolzmann method), and two mini-apps (LULESH and miniFE) on two heterogeneous systems with different hardware capabilities. This new Kokkos feature provides high accelerations of up to 35× thanks to automatic and transparent device selection.
Lee, Seyong↗
Accelerating an iterative eigensolver for nuclear structure configuration interaction calculations on GPUs using OpenACC
Not Available
97 MATHEMATICS AND COMPUTING↗
Optimization with the OpenACC-to-FPGA framework on the Arria 10 and Stratix 10 FPGAs
The reconfigurable computing paradigm with field programmable gate arrays (FPGAs) has received renewed interest in the high-performance computing field due to FPGAs’ unique combination of performance and energy efficiency. However, difficulties in programming and optimizing FPGAs have prevented them from being widely accepted as general-purpose computing devices. In accelerator-based heterogeneous computing, portability across diverse heterogeneous devices is also an important issue, but the unique architectural features in FPGAs make this difficult to achieve. To address these issues, a directive-based, high-level FPGA programming and optimization framework was previously developed. In this work, developed optimizations were combined holistically using the directive-based approach to show that each individual benchmark requires a unique set of optimizations to maximize performance. We perform this exploration on Intel Arria 10 and Stratix 10 FPGAs. We also explored the relationships between performance, resource usages, and compilation times, and investigated implications for performance portability. Finally, we present an initial evaluation of a real-world proxy application, LULESH.
97 MATHEMATICS AND COMPUTING↗
In-Depth Optimization with the OpenACC-to-FPGA Framework on an Arria 10 FPGA
The reconfigurable computing paradigm that uses field programmable gate arrays (FPGAs) has received renewed interest in the high-performance computing field due to FPGAs' unique combination of performance and energy efficiency. However, difficulties in programming and optimizing FPGAs have prevented them from being widely accepted as general-purpose computing devices. In accelerator-based heterogeneous computing, portability across diverse heterogeneous devices is also an important issue, but the unique architectural features in FPGAs make this difficult to achieve. To address these issues, a directive-based, high-level FPGA programming and optimization framework was previously developed. In this work, developed optimizations were combined holistically using the directive-based approach to show that each individual benchmark requires a unique set of optimizations to maximize performance. The relationships between FPGA resource usages and runtime performance were also explored.
Lambert, Jacob↗