Engineering topics

Yeung, P. K.

Publications and source records attributed to Yeung, P. K..

6 records

DOE OSTI2024

GPU-enabled extreme-scale turbulence simulations: Fourier pseudo-spectral algorithms at the exascale using OpenMP offloading

Fourier pseudo-spectral methods for nonlinear partial differential equations are of wide interest in many areas of advanced computational science, including direct numerical simulation of three-dimensional (3-D) turbulence governed by the Navier-Stokes equations in fluid dynamics. This paper presents a new capability for simulating turbulence at a new record resolution up to 35 trillion grid points, on the world's first exascale computer, Frontier, comprising AMD MI250x GPUs with HPE's Slingshot interconnect and operated by the US Department of Energy's Oak Ridge Leadership Computing Facility (OLCF). Key programming strategies designed to take maximum advantage of the machine architecture involve performing almost all computations on the GPU which has the same memory capacity as the CPU, performing all-to-all communication among sets of parallel processes directly on the GPU, and targeting GPUs efficiently using OpenMP offloading for intensive number-crunching including 1-D Fast Fourier Transforms (FFT) performed using AMD ROCm library calls. With 99% of computing power on Frontier being on the GPU, leaving the CPU idle leads to a net performance gain via avoiding the overhead of data movement between host and device except when needed for some I/O purposes. Memory footprint including the size of communication buffers for MPI_ALLTOALL is managed carefully to maximize the largest problem size possible for a given node count. Detailed performance data including separate contributions from different categories of operations to the elapsed wall time per step are reported for five grid resolutions, from 2048 3 on a single node to 32768 3 on 4096 or 8192 nodes out of 9408 on the system. Both 1D and 2D domain decompositions which divide a 3D periodic domain into slabs and pencils respectively are implemented. The present code suite (labeled by the acronym GESTS, GPUs for Extreme Scale Turbulence Simulations) achieves a figure of merit (in grid points per second) exceeding goals set in the Center for Accelerated Application Readiness (CAAR) program for Frontier. The performance attained is highly favorable in both weak scaling and strong scaling, with notable departures only for 2048 3 where communication is entirely intra-node, and for 32768 3 , where a challenge due to small message sizes does arise. Communication performance is addressed further using a lightweight test code that performs all-to-all communication in a manner matching the full turbulence simulation code. Performance at large problem sizes is affected by both small message size due to high node counts as well as dragonfly network topology features on the machine, but is consistent with official expectations of sustained performance on Frontier. Overall, although not perfect, the scalability achieved at the extreme problem size of 32768 3 (and up to 8192 nodes — which corresponds to hardware rated at just under 1 exaflop/sec of theoretical peak computational performance) is arguably better than the scalability observed using prior state-of-the-art algorithms on Frontier's predecessor machine (Summit) at OLCF. New science results for the study of intermittency in turbulence enabled by this code and its extensions are to be reported separately in the near future.

3D fast Fourier transform↗

NASA NTRS1998

Numerical Study of Rotating Turbulence with External Forcing

Direct numerical simulation at 256(exp 3) resolution have been carried out to study the response of isotropic turbulence to the concurrent effects of solid-body rotation and numerical forcing at the large scales. Because energy transfer to the smaller scales is weakened by rotation, energy input from forcing gradually builds up at the large scales, causing the overall kinetic energy to increase. At intermediate wavenumbers the energy spectrum undergoes a transition from a limited k(exp -5/3) inertial range to k(exp -2) scaling recently predicted in the literature. Although the Reynolds stress tensor remains approximately isotropic and three-components, evidence for anisotropy and quasi- two-dimensionality in length scales and spectra in different velocity components and directions is strong. The small scales are found to deviate from local isotropy, primarily as a result of anisotropic transfer to the high wavenumbers. To understand the spectral dynamics of this flow we study the detailed behavior of nonlinear triadic interactions in wavenumber space. Spectral transfer in the velocity component parallel to the axis of rotation is qualitatively similar to that in non-rotating turbulence; however the perpendicular component is characterized by a greatly suppressed energy cascade at high wavenumber and a local reverse transfer at the largest scales. The broader implications of this work are briefly addressed.

Yeung, P. K.↗

NASA NTRS1997

On the Universality of the Kolmogorov Constant in Numerical Simulations of Turbulence

Motivated by a recent survey of experimental data, we examine data on the Kolmogorov spectrum constant in numerical simulations of isotropic turbulence, using results both from previous studies and from new direct numerical simulations over a range of Reynolds numbers (up to 240 on the Taylor scale) at grid resolutions up to 512(exp 3). It is noted that in addition to k(exp -5/3) scaling, identification of a true inertial range requires spectral isotropy in the same wavenumber range. We found that a plateau in the compensated three-dimensional energy spectrum at k(eta) approx. = 0.1 - -0.2, commonly used to infer the Kolmogorov constant from the compensated three-dimensional energy spectrum, actually does not represent proper inertial range behavior. Rather, a proper, if still approximate, inertial range emerges at k(eta) approx. = 0.02 - 0.05 when R(sub lambda) increases beyond 140. The new simulations indicate proportionality constants C(sub 1) and C in the one- and three-dimensional energy spectra respectively about 0.60 and 1.62. If the turbulence were perfectly isotropic then use of isotropy relations in wavenumber space (C(sub 1) = 18/55 C) would imply that C(sub 1) approx. = 0.53 for C = 1.62, in excellent agreement with experiments. However the one- and three-dimensional estimates are not fully consistent, because of departures (due to numerical and statistical limitations) from isotropy of the computed spectra at low wavenumbers. The inertial scaling of structure functions in physical space is briefly addressed. Since DNS is still restricted to moderate Reynolds numbers, an accurate evaluation of the Kolmogorov constant is very difficult. We focus on providing new insights on the interpretation of Kolmogorov 1941 similarity in the DNS literature and do not consider issues pertaining to the refined similarity hypotheses of Kolmogorov (K62).

Yeung, P. K.↗

NASA NTRS1996

Analysis of Subgrid Eddy Diffusivity in Turbulence Using Direct Numerical Simulation Databases

Direct numerical simulation data in isotropic turbulence at Taylor-scale Reynolds number 90 are used to compute subgrid transfer and spectral eddy diffusivities for passive scalars at Schmidt numbers 1 and 1/8. The results are consistent with spectral transfer processes dominated by a local forward cascade in wavenumber space which is a consequence of moderately nonlocal interactions in the scalar fields. Data decompositions show that resolvable-subgrid interactions make (in general) substantially stronger contributions than fully-subgrid interactions to the eddy diffusivity, which has a non-negligible inverse cascade, or backscatter, component. The eddy diffusivity has a strong cusp at wavenumbers close to the cutoff value separating resolvable and subgrid ranges. At low wavenumbers the eddy diffusivity is generally small, but there the details depend on Schmidt number. The behavior of the subgrid turbulent Schmidt number (the ratio of effective eddy viscosity to eddy diffusivity) is observed to depend strongly on Schmidt number. In interpreting the results at a given spectral cutoff it is important to note that the spectrum of less diffusive scalars (of higher Schmidt number) has more high-wavenumber content.

Yeung, P. K.↗

NASA NTRS1994

Analysis of subgrid models using direct and large-eddy simulations of isotropic turbulence

Direct and large eddy simulations of forced and decaying isotropic turbulence have been performed using a pseudospectral and a finite-difference code. Subgrid models that include a one-equation subgrid kinetic energy model with and without a stochastic backscatter forcing term and a new scale similarity model have been analyzed in both Fourier space and physical space. The Fourier space analysis showed that the energy transfer across the cutoff wavenumber k(sub c) is dominated by local interaction. The correlation between the exact and the modeled (by a spectral eddy viscosity) nonlinear terms and the subgrid energy transfer in physical space was found to be quite low. In physical space, a similar correlation analysis was carried out using top hat filtering. Results show that the subgrid stress and the energy flux predicted by the subgrid models correlates very well with the exact data. The scale similarity model showed very high correlation for reasonable grid resolution. However, with decrease in grid resolution, the scale similarity model became more uncorrelated, when compared to the kinetic energy subgrid model. The subgrid models were then used for large-eddy simulations for a range of Reynolds number. It was determined that the dissipation was modeled poorly and that the correlation with the exact results was quite low for all the models. In general, for coarse grid resolution, the scale similarity model consistently showed very low correlation while the kinetic energy model showed a relatively higher correlation. These results suggest that to use the scale similarity model relatively fine grid resolution may be required, whereas, the kinetic energy model could be used even in coarse grid.

Menon, S.↗

NASA NTRS1994

Scale disparity and spectral transfer in anisotropic numerical turbulence

To study the effect of cancellations within long-range interactions on local isotropy at the small scales, we calculate explicitly the degree of cancellation in distant interactions in the simulations of Yeung & Brasseur and Yeung, Brasseur & Wang using the single scale disparity parameter 's' developed by Zhou. In the simulations, initially isotropic simulated turbulence was subjected to coherent anisotropic forcing at the large scales and the smallest scales were found to become anisotropic as a consequence of direct large-small scale couplings. We find that the marginally distant interactions in the simulation do not cancel out under summation and that the development of small-scale anisotropy is indeed a direct consequence of the distant triadic group, as argued by Yeung, et. al. A reduction of anisotropy at later times occurs as a result of the isotropizing influences of more local energy-cascading triadic interactions. Nevertheless, the local-to-nonlocal triadic group persists as an isotropizing influence at later times. We find that, whereas long-range interactions, in general, contribute little to net energy transfer into or out of a high wavenumber shell k, the anisotropic transfer of component energy within the shell increases with increasing scale separations. These results are consistent with results by Zhou, and Brasseur & Wei, and suggest that the anisotropizing influences of long range interactions should persist to higher Reynolds numbers. The residual effect of the forced distant group in this low-Reynolds number simulation is found to be forward cascading, on average.

Zhou, YE↗