Search NASA⌕ Search

Engineering topics

Lo, Li-Ta

Publications and source records attributed to Lo, Li-Ta.

Visualization at exascale: Making it all work with VTK-m

The VTK-m software library enables scientific visualization on exascale-class supercomputers. Exascale machines are particularly challenging for software development in part because they use GPU accelerators to provide the vast majority of their computational throughput. Algorithmic designs for GPUs and GPU-centric computing often deviate from those that worked well on previous generations of high-performance computers that relied on traditional CPUs. Fortunately, VTK-m provides scientific visualization algorithms for GPUs and other accelerators. VTK-m also provides a framework that simplifies the implementation of new algorithms and adds a porting layer to work across multiple processor types. This paper describes the main challenges encountered when making scientific visualization available at exascale. Here, we document the surprises and obstacles faced when moving from pre-exascale platforms to the final exascale designs and the performance on those systems including scaling studies on Frontier, an exascale machine with over 37,000 AMD GPUs. We also report on the integration of VTK-m with other exascale software technologies. Finally, we show how VTK-m helps scientific discovery for applications such as fusion and particle acceleration that leverage an exascale supercomputer.

97 MATHEMATICS AND COMPUTING↗

VTK-m: Visualization for the Exascale Era and Beyond

A recent trend in modern high-performance computing is the increasing use of hybrid architectures, where the vast majority of performance comes from accelerators. Modern accelerators are based on Graphics Processing Units (GPU) that contain many low power cores that in their aggregate provides an extremely high computation rate. Current and future CPU processors are requiring more explicit parallelism as each successive version of the hardware packs in more cores, and technologies like hyperthreading and vector operations require even more parallel processing to leverage each core’s full potential. As an example, the Frontier supercomputer installed at Oak Ridge National Laboratories recently hit a record breaking 1.1 exaflops1 on the LINPACK HPC benchmark [Shoemaker 2022]. The system contains 37632 AMD MI250x GPUs which requires more than half a billion threads to keep the system fully utilized [Khizeran 2022].VTK-m is a toolkit of scientific visualization algorithms for these emerging processor architectures. VTK-m supports the fine-grained concurrency for data analysis and visualization algorithms required to drive extreme scale computing by providing abstract models for data and execution that can be applied to a variety of algorithms across many different processor architectures.

Bolstad, Mark↗

VTK-m v. 2.0

VTK-m is a toolkit of scientific visualization algorithms for emerging processor architectures. VTK-m supports the fine-grained concurrency for data analysis and visualization algorithms required to drive extreme scale computing by providing abstract models for data and execution that can be applied to a variety of algorithms across many different processor architectures.

Moreland, Kenneth [Oak Ridge National Lab. (ORNL),↗

The VTK-m Users' Guide (V.1.9)

High-performance computing relies on ever finer threading. Advances in processor technology include ever greater numbers of cores, hyperthreading, accelerators with integrated blocks of cores, and special vectorized instructions, all of which require more software parallelism to achieve peak performance. Traditional visualization solutions cannot support this extreme level of concurrency. Extreme scale systems require a new programming model and a fundamental change in how we design algorithms. To address these issues we created VTK-m: the visualization toolkit for multi-/many-core architectures. VTK-m supports a number of algorithms and the ability to design further algorithms through a top-down design with an emphasis on extreme parallelism. VTK-m also provides support for finding and building links across topologies, making it possible to perform operations that determine manifold surfaces, interpolate generated values, and find adjacencies. Although VTK-m provides a simplified high-level interface for programming, its template-based code removes the overhead of abstraction. VTK-m simplifies the development of parallel scientific visualization algorithms by providing a framework of supporting functionality that allows developers to focus on visualization operations. Consider the listings in Figure 1.1 that compares the size of the implementation for the Marching Cubes algorithm in VTK-m with the equivalent reference implementation in the CUDA software development kit. Because VTK-m internally manages the parallel distribution of work and data, the VTK-m implementation is shorter and easier to maintain. Additionally, VTK-m provides data abstractions not provided by other libraries that make code written in VTK-m more versatile.

97 MATHEMATICS AND COMPUTING↗

The Importance of Scientific Visualization on Novel Hardware

Innovation in HPC hardware and adoption of heterogeneous systems has led to a variety of unique programming models. This has led to a challenge for scientific visualization software (and indeed all HPC software) to take full advantage of recent generations of supercomputing. Edge computing, where hardware is specialized for the needs of the particular application, exacerbates the problem. VTK-m has had many successes on this front by providing device-agnostic algorithms that compare favorably to implementations written to specific devices as shown in Table 1. However, VTK-m has focused mostly on GPU and traditional CPU multicore technology. There are numerous processor technologies, both existing and potential future, that are not being addressed by current R&D efforts.

97 MATHEMATICS AND COMPUTING↗

The Ristra Project: FY20/21 Milestone Report

The ASC Advanced Technology Development and Mitigation (ATDM) sub-program was established in 2014 to develop new simulation tools operating on exascale-class computers to serve NNSA (see Appendix B). Over the course of ATDM, LANL management have set a strategy for exascale-class application codes that follows two supportive and mutually risk-mitigating paths: evolution for established production integrated design codes (IDCs) – with a strong pedigree within the user community – based upon existing programming paradigms(MPI+X); and a new start ATDM project, Ristra, a high-risk/high-reward push for a next-generation multi-physics, multi-scale simulation toolkit based on emerging advanced programming systems(with an initial focus on data-flow task-based models exemplified by Legion). The role of Ristra as the high-risk/high-reward path for LANL’s codes was fully consistent with the goals of ATDM as described in Appendix B, in particular its emphasis on evolving ASC capabilities through novel computing programming models and computing technologies.

97 MATHEMATICS AND COMPUTING↗

Corrigendum and Addendum: Helium flux effects on bubble growth and surface morphology in plasma-facing tungsten from large-scale molecular dynamics simulations (2019 Nucl. Fusion 59 066035)

Two of the simulations discussed in a prior article (Hammond et al 2019 Nucl. Fusion 59 066035) were affected by a simulation glitch. We repeated the affected calculations and discuss them here. Here, the overall conclusions are essentially unchanged, though the details are different. In particular, observations that we referred to as ‘concerted bursting’ were caused primarily by non-physical heating and cooling applied by the thermostat after most atoms’ velocities were deleted (for reasons that are not known for certain). The phenomenon of one bubble bursting and causing another nearby bubble to burst does exist, though its effects are much less spectacular in the absence of non-physical driving forces. The observation of an interconnected network of sub-surface cavities formed by burst bubbles is real, and the observation of holes on the surface 1–2 nm in diameter is also confirmed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Helium flux effects on bubble growth and surface morphology in plasma-facing tungsten from large-scale molecular dynamics simulations

We investigate helium flux effects on helium transport and surface evolution in plasma-facing tungsten using molecular dynamics. The simulations span two orders of magnitude, from ITER-relevant levels to those more typical of simulations published to date. Simulation times of up to 2.5 µs (corresponding to actual fluences of 1.4 x 10 21 m -2 ) are achieved, revealing concerted bubble-bursting events that are responsible for significant and very sudden changes in surface morphology. The depth distribution of helium depends very strongly on helium flux: helium self-trapping becomes more probable near the surface at high flux, and a layer of near-surface bubbles forms. Helium retention prior to the onset of bubble bursting is also substantially lower at low flux than it is at high flux. Surface features at low fluence are correlated with the positions of bubbles, but at high fluence, bubbles tend to coalesce, venting to the surface at one or more locations and leaving large interconnected cavities below the surface. Ruptured bubbles may serve as pathways deeper into the material, allowing helium to bypass the layer of near-surface bubbles and fill deeper, potentially much larger, bubbles that can produce more substantial surface features. Deeper bubbles also emit prismatic dislocation loops that can fill in cavities closer to the surface. Our results suggest that nearly all molecular dynamics simulations published to date are hampered by finite-size effects, and that helium flux is a very important parameter in determining the behavior of helium in plasma-facing components.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Portable Data-Parallel Surface Reconstruction on a Uniform Rectilinear Grid

With the increasing heterogeneity and on-node parallelism of high-performance computing hardware, a major challenge is to develop portable and efficient algorithms and software. In this work, we present our implementation of a portable code to perform surface reconstruction using NVIDIA's Thrust library. Surface reconstruction is a technique commonly used in volume tracking methods for simulations of multimaterial flow with interfaces. Here, we have designed a 3D mesh data structure that is easily mapped to the 1D vectors used by Thrust and at the same time is simple to use and uses familiar data structure terminology (such as cells, faces, vertices, and edges). With this new data structure in place, we have implemented a piecewise linear interface reconstruction algorithm in 3 dimensions that effectively exploits the symmetry present in a uniform rectilinear computational cell. Finally, we report performance results, which show that a single implementation of these algorithms can be compiled to multiple backends (specifically, multi-core CPUs, NVIDIA GPUs, and Intel Xeon Phi processors), making efficient use of the available parallelism on each. We also compare performance of our implementation to a legacy FORTRAN implementation in Message Passing Interface (MPI) and show performance parity on single and multi-core CPU and achieved good parallel speed-ups on GPU. In conclusion, our research demonstrates the advantage of performance portability of the underlying data-parallel programming model.

97 MATHEMATICS AND COMPUTING↗

Cluster analysis of accelerated molecular dynamics simulations: A case study of the decahedron to icosahedron transition in Pt nanoparticles

Modern molecular-dynamics-based techniques are extremely powerful to investigate the dynamical evolution of materials. With the increase in sophistication of the simulation techniques and the ubiquity of massively parallel computing platforms, atomistic simulations now generate very large amounts of data, which has to be carefully analyzed in order to reveal key features of the underlying trajectories, including the nature and characteristics of the relevant reaction pathways. We show that clustering algorithms, such as the Perron Cluster Cluster Analysis, can provide reduced representations that greatly facilitate the interpretation of complex trajectories. To illustrate this point, clustering tools are used to identify the key kinetic steps in complex Accelerated Molecular Dynamics trajectories exhibiting shape uctuations in Pt nanoclusters. This analysis provides an easily-interpretable coarse representation of the reaction pathways in terms of a handful of clusters, in contrast to the raw trajectory that contains thousands of unique states and tens of thousands of transitions.

74 ATOMIC AND MOLECULAR PHYSICS↗

Simple rANDom wALk simulation

SANDAL is a particle-mesh simulation mini-app. The simple simulated physics propagates a set of particles in a constant, 2D wind field with Gaussian turbulence. SANDAL demonstrates the feasibility of formulating a computational physics problem using an alternative, modern design. Specifically, it is implemented using relational tables and queries, rather than array-based data model. It also is implemented with functional language design and cloud deployment, Scala and Apache Spark, rather than Fortran and MPI+X.

Lo, Li-Ta↗