Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel multigrid”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Task 7: ADPAC User's Manual

The overall objective of this study was to develop a 3-D numerical analysis for compressor casing treatment flowfields. The current version of the computer code resulting from this study is referred to as ADPAC (Advanced Ducted Propfan Analysis Codes-Version 7). This report is intended to serve as a computer program user's manual for the ADPAC code developed under Tasks 6 and 7 of the NASA Contract. The ADPAC program is based on a flexible multiple- block grid discretization scheme permitting coupled 2-D/3-D mesh block solutions with application to a wide variety of geometries. Aerodynamic calculations are based on a four-stage Runge-Kutta time-marching finite volume solution technique with added numerical dissipation. Steady flow predictions are accelerated by a multigrid procedure. An iterative implicit algorithm is available for rapid time-dependent flow calculations, and an advanced two equation turbulence model is incorporated to predict complex turbulent flows. The consolidated code generated during this study is capable of executing in either a serial or parallel computing mode from a single source code. Numerous examples are given in the form of test cases to demonstrate the utility of this approach for predicting the aerodynamics of modem turbomachinery configurations.

Hall, E. J.↗

Deploy threading in Nalu solver stack

The goal of the ExaWind project is to enable predictive simulations of wind farms composed of many MW-scale turbines situated in complex terrain. Predictive simulations will require computational fluid dynamics (CFD) simulations for which the mesh resolves the geometry of the turbines, and captures the rotation and large deflections of blades. Whereas such simulations for a single turbine are arguably petascale class, multi-turbine wind farm simulations will require exascale-class resources. The primary code in the ExaWind project is Nalu, which is an unstructured-grid solver for the acousticallyincompressible Navier-Stokes equations, and mass continuity is maintained through pressure projection. The model consists of the mass-continuity Poisson-type equation for pressure and a momentum equation for the velocity. For such modeling approaches, simulation times are dominated by linear-system setup and solution for the continuity and momentum systems. For the ExaWind challenge problem, the moving meshes greatly affect overall solver costs as re-initialization of matrices and re-computation of preconditioners is required at every time step In this Milestone, we examine the effect of threading on the solver stack performance against flat-MPI results obtained from previous milestones using Haswell performance data full-turbine simulations. Whereas the momentum equations are solved only with the Trilinos solvers, we investigate two algebraic-multigrid preconditioners for the continuity equations: Trilinos/Muelu and HYPRE/BoomerAMG. These two packages embody smoothed-aggregation and classical Ruge-Stiiben AMG methods, respectively. In our FY18 Q2 report, we described our efforts to improve setup and solve of the continuity equations under flat-MPI parallelism. While significant improvement was demonstrated in the solve phase, setup times remained larger than expected. Starting with the optimized settings described in the Q2 report, we explore here simulation performance where OpenMP threading is employed in the solver stack. For Trilinos, threading is acheived through the Kokkos abstraction where, whereas HYPRE/BoomerAMG employs straight OpenMP. We examined results for our mid-resolution baseline turbine simulation configuration (229M DOF). Simulations on 2048 Haswell cores explored the effect of decreasing the number of MPI ranks while increasing the number of threads. Both HYPRE and Trilinos exhibited similar overal solution times, and both showed dramatic increases in simulation time in the shift from MPI ranks to OpenMP threads. This increase is attributed to the large amount of work per MPI rank starting at the single-thread configuration. Decreasing MPI ranks, while increasing threads, may be increasing simulation time due to thread synchronization and start-up overhead contributing to the latency and serial time in the model. These result showed that an MPI+OpenMP parallel decomposition will be more effective as the amount per MPI rank computation per MPI rank decreases and the communication latency increases. This idea was demonstrated in a strong scaling study of our low-resolution baseline model (29M DOF) with the Trilinos-HYPRE configuration. While MPI-only results showed scaling improvement out to about 1536 cores, engaging threading carried scaling improvements out to 4128 cores — roughly 7000 DOF per core. This is an important result as improved strong scaling is needed for simulations to be executed over sufficiently long simulated durations (i.e., for many timesteps). In addition to threading work described above, the team examined solver-performance improvements by exploring communication-overhead in the HYPRE-GMRES implementation through a communicationoptimal- GMRE algorithm (CO-GMRES), and offloading compute-intensive solver actions to GPUs. To those ends, a HYPRE mini-app was allow us to easily test different solver approaches and HYPRE parameter settings without running the entire Nalu code. With GPU acceleration on the Summitdev supercomputer, a 20x speedup was achieved for the overall preconditioner and solver execution time for the mini-app. A study on Haswell processors showed that CO-GMRES provides benefits as one increases MPI ranks.

17 WIND ENERGY↗

RIACS

Topics considered include: high-performance computing; cognitive and perceptual prostheses (computational aids designed to leverage human abilities); autonomous systems. Also included: development of a 3D unstructured grid code based on a finite volume formulation and applied to the Navier-stokes equations; Cartesian grid methods for complex geometry; multigrid methods for solving elliptic problems on unstructured grids; algebraic non-overlapping domain decomposition methods for compressible fluid flow problems on unstructured meshes; numerical methods for the compressible navier-stokes equations with application to aerodynamic flows; research in aerodynamic shape optimization; S-HARP: a parallel dynamic spectral partitioner; numerical schemes for the Hamilton-Jacobi and level set equations on triangulated domains; application of high-order shock capturing schemes to direct simulation of turbulence; multicast technology; network testbeds; supercomputer consolidation project.

Oliger, Joseph↗

A single-assignment language in a distributed memory multiprocessor

The implementation of the single-assignment programming language SISAL (McGraw et al., 1985) on a Symult 2010 parallel computer is described. The advantages of single-assignment languages over imperative languages in a multiprocessor environment are reviewed; the characteristics of SISAL are summarized; the program-graph generation and dynamic data partitioning procedures are explained; and the application of SISAL in constructing a concurrent iterative multigrid algorithm is discussed in detail and illustrated with diagrams.

Evripidou, P.↗

Parallel-in-Time Methods for Method-of-Lines Discretizations of Nonlinear Hyperbolic PDEs and Systems (Final Report)

The work for the subcontract is situated in the area of parallel-in-time integration for hyperbolic partial differential equations (PDEs). Parallel-in-time integration is an active area of research due to its ability to enable faster numerical simulations for applications throughout many areas of science. The work in this subcontract builds on a variety of results that were obtained, as part of the work performed for Subcontract No. B648355, for the Multigrid Reduction-in-Time (MGRIT) method from [1] applied to hyperbolic PDEs. This subcontract extends these results further to more efficient methods and to the case of method-of-lines discretizations for nonlinear hyperbolic PDES and systems of PDEs. The following is a summary of the research performed and results achieved during milestone periods 1, 2 and 3 by the PI (Hans De Sterck) and Postdoctoral Research Associate (Oliver Krzysik), for required tasks 1-4 (as listed in the Statement of Work): Research over the previous year has been split into three main projects: (i) solution of acoustic equation system; (ii) solution of nonlinear scalar hyperbolic PDEs; (iii) solution of nonlinear hyperbolic systems of PDEs.

97 MATHEMATICS AND COMPUTING↗

A Multilevel in Space and Energy Solver for 3-D Multigroup Diffusion and Coarse-Mesh Finite Difference Eigenvalue Problems

The Multilevel in Space and Energy Diffusion (MSED) method accelerates the iterative convergence of multigroup diffusion eigenvalue problems by performing work on lower-order equations with only one group and/or coarser spatial grids. It consists of two primary components: (1) a grey (one-group) diffusion eigenvalue problem that is solved via Wielandt-shifted power iteration (PI) and (2) a multigrid-in-space linear solver. In previous work, the efficiency of MSED was verified using Fourier analysis and numerical results from a one-dimensional multigroup diffusion code. Since that work, MSED has been implemented as a solver for the coarse-mesh finite difference (CMFD) system in the three-dimensional Michigan Parallel Characteristics Transport (MPACT) code. In this paper, the results from the implementation of MSED in MPACT are introduced, and the changes needed to make MSED more suitable for MPACT are described. For problems without feedback, the conclusions in this paper show that MSED can reduce the CMFD run time by an order of magnitude and the overall run time by a factor of 2 to 3 compared to the default CMFD solver in MPACT [PI with the generalized minimal residual (GMRES) method]. For problems with feedback, the convergence of the outer Picard iteration scheme is worsened by the well-converged CMFD solutions produced by the standard MSED method. To overcome this unintuitive deficiency, MSED may be run with looser convergence criteria (a modified version of the MSED method called MSED-L) to circumvent the issue until the multiphysics iteration in MPACT is improved. Results show that MSED-L can reduce the CMFD run time in MPACT by an order of magnitude, without negatively impacting the outer Picard iteration scheme.

3-D↗

Layout optimization with algebraic multigrid methods

Finding the optimal position for the individual cells (also called functional modules) on the chip surface is an important and difficult step in the design of integrated circuits. This paper deals with the problem of relative placement, that is the minimization of a quadratic functional with a large, sparse, positive definite system matrix. The basic optimization problem must be augmented by constraints to inhibit solutions where cells overlap. Besides classical iterative methods, based on conjugate gradients (CG), we show that algebraic multigrid methods (AMG) provide an interesting alternative. For moderately sized examples with about 10000 cells, AMG is already competitive with CG and is expected to be superior for larger problems. Besides the classical 'multiplicative' AMG algorithm where the levels are visited sequentially, we propose an 'additive' variant of AMG where levels may be treated in parallel and that is suitable as a preconditioner in the CG algorithm.

Regler, Hans↗

ADPAC v1.0: User's Manual

The overall objective of this study was to evaluate the effects of turbulence models in a 3-D numerical analysis on the wake prediction capability. The current version of the computer code resulting from this study is referred to as ADPAC v7 (Advanced Ducted Propfan Analysis Codes -Version 7). This report is intended to serve as a computer program user's manual for the ADPAC code used and modified under Task 15 of NASA Contract NAS3-27394. The ADPAC program is based on a flexible multiple-block and discretization scheme permitting coupled 2-D/3-D mesh block solutions with application to a wide variety of geometries. Aerodynamic calculations are based on a four-stage Runge-Kutta time-marching finite volume solution technique with added numerical dissipation. Steady flow predictions are accelerated by a multigrid procedure. Turbulence models now available in the ADPAC code are: a simple mixing-length model, the algebraic Baldwin-Lomax model with user defined coefficients, the one-equation Spalart-Allmaras model, and a two-equation k-R model. The consolidated ADPAC code is capable of executing in either a serial or parallel computing mode from a single source code.

Hall, Edward J.↗

Mapping implicit spectral methods to distributed memory architectures

Spectral methods were proven invaluable in numerical simulation of PDEs (Partial Differential Equations), but the frequent global communication required raises a fundamental barrier to their use on highly parallel architectures. To explore this issue, a 3-D implicit spectral method was implemented on an Intel hypercube. Utilization of about 50 percent was achieved on a 32 node iPSC/860 hypercube, for a 64 x 64 x 64 Fourier-spectral grid; finer grids yield higher utilizations. Chebyshev-spectral grids are more problematic, since plane-relaxation based multigrid is required. However, by using a semicoarsening multigrid algorithm, and by relaxing all multigrid levels concurrently, relatively high utilizations were also achieved in this harder case.

Overman, Andrea L.↗

MueLu User's Guide

This is the official user guide for MUELU multigrid library in Trilinos version 12.13 (Dev). This guide provides an overview of MUELU, its capabilities, and instructions for new users who want to start using MUELU with a minimum of effort. Detailed information is given on how to drive MUELU through its XML interface. Links to more advanced use cases are given. This guide gives information on how to achieve good parallel performance, as well as how to introduce new algorithms Finally, readers will find a comprehensive listing of available MUELU options. Any options not documented in this manual should be considered strictly experimental.

97 MATHEMATICS AND COMPUTING↗

Effective Boundary Treatment for the Biharmonic Dirichlet Problem

The biharmonic equation can be rewritten as a system of two Poisson equations. Multigrid solution of this system is expected to converge with the same amount of work as solving two Poisson equations, requiring less than 70 floating point operations (scalar multiply or addition) per fine grid point to reach a solution using an FMG algorithm. For periodic boundary conditions, this goal is attained by simple, straightforward application of multigrid. For Dirichlet boundary conditions, however, convergence is impeded by poor interaction with the boundaries. Attempts to overcome the slowness without specifically addressing the boundaries have resulted in multigrid algorithms not attaining the Poisson convergence rate. We present three methods of boundary treatment with which full multigrid efficiency can be obtained. All implement an approach described by Brandt, concentrating some additional effort near the boundary. The first approach simply adds a number of relaxation sweeps over points close to the boundary. The second uses joint relaxation on near-boundary points. The third method takes something from each of the first two methods, resulting in a solver more suitable for highly parallel applications.

Brandt, A.↗

Performance of fully-coupled algebraic multigrid preconditioners for large-scale VMS resistive MHD

Here, we explore the current performance and scaling of a fully-implicit stabilized unstructured finite element (FE) variational multiscale (VMS) capability for large-scale simulations of 3D incompressible resistive magnetohydrodynamics (MHD). The large-scale linear systems that are generated by a Newton nonlinear solver approach are iteratively solved by preconditioned Krylov subspace methods. The efficiency of this approach is critically dependent on the scalability and performance of the algebraic multigrid preconditioner. Our study considers the performance of the numerical methods as recently implemented in the second-generation Trilinos implementation that is 64-bit compliant and is not limited by the 32-bit global identifiers of the original Epetra-based Trilinos. The study presents representative results for a Poisson problem on 1.6 million cores of an IBM Blue Gene/Q platform to demonstrate very large-scale parallel execution. Additionally, results for a more challenging steady-state MHD generator and a transient solution of a benchmark MHD turbulence calculation for the full resistive MHD system are also presented. These results are obtained on up to 131,000 cores of a Cray XC40 and one million cores of a BG/Q system.

97 MATHEMATICS AND COMPUTING↗

Simulation of 24,000 Electron Dynamics: Real-Time Time-Dependent Density Functional Theory (TDDFT) with the Real-Space Multigrids (RMG)

Here, we present the theory, implementation, and benchmarking of a real-time time-dependent density functional theory (RT-TDDFT) module within the RMG code, designed to simulate the electronic response of molecular systems to external perturbations. Our method offers insights into nonequilibrium dynamics and excited states across a diverse range of systems, from small organic molecules to large metallic nanoparticles. Benchmarking results demonstrate excellent agreement with established TDDFT implementations and showcase the superior stability of our time integration algorithm, enabling long-term simulations with minimal energy drift. The scalability and efficiency of RMG on massively parallel architectures allow for simulations of complex systems, such as plasmonic nanoparticles with thousands of atoms. Future extensions, including nuclear and spin dynamics, will broaden the applicability of this RT-TDDFT implementation, providing a powerful toolset for studies of photoactive materials, nanoscale devices, and other systems where real-time electronic dynamics is essential.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

End-to-end GPU acceleration of low-order-refined preconditioning for high-order finite element discretizations

In this article, we present algorithms and implementations for the end-to-end GPU acceleration of matrix-free low-order-refined preconditioning of high-order finite element problems. The methods described here allow for the construction of effective preconditioners for high-order problems with optimal memory usage and computational complexity. The preconditioners are based on the construction of a spectrally equivalent low-order discretization on a refined mesh, which is then amenable to, for example, algebraic multigrid preconditioning. The constants of equivalence are independent of mesh size and polynomial degree. For vector finite element problems in H(curl) and H(div) (e.g., for electromagnetic or radiation diffusion problems), a specially constructed interpolation–histopolation basis is used to ensure fast convergence. Detailed performance studies are carried out to analyze the efficiency of the GPU algorithms. The kernel throughput of each of the main algorithmic components is measured, and the strong and weak parallel scalability of the methods is demonstrated. The different relative weighting and significance of the algorithmic components on GPUs and CPUs is discussed. Results on problems involving adaptively refined nonconforming meshes are shown, and the use of the preconditioners on a large-scale magnetic diffusion problem using all spaces of the finite element de Rham complex is illustrated.

97 MATHEMATICS AND COMPUTING↗

Preconditioning a Newton-Krylov solver for all-speed melt pool flow physics

n this paper, we introduce a multigrid block-based preconditioner for solving linear systems arising from a Discontinuous Galerkin discretization of the all-speed Navier-Stokes equations with phase change. The equations are discretized in conservative form with a reconstructed Discontinuous Galerkin (rDG) method and integrated with fully-implicit time discretization schemes. To robustly converge the numerically stiff systems, we use the Newton-Krylov framework with a primitive-variable formulation (pressure, velocity, and temperature), which is better conditioned than the conservative-variable form at low-Mach number. In the limit of large acoustic CFL number and viscous Fourier number, there is a strong coupling between the velocity-pressure system and the linear systems become non-diagonally dominant. To effectively solve these ill-conditioned systems, an approximate block factorization preconditioner is developed, which uses the Schur complement to reduce a 3 x 3 block system into a sequence of two 2 x 2 block systems: velocity-pressure,v P , and velocity-temperature, v T . We compare the performance of the v P -v T Schur complement preconditioner to classic preconditioning strategies: monolithic algebraic multigrid (AMG), element-block SOR, and primitive variable block Gauss-Seidel. The performance of the preconditioned solver is investigated in the limit of large CFL and Fourier numbers for low-Mach lid-driven cavity flow, Rayleigh-Bénard melt convection, compressible internally heated convection, and 3D laser-induced melt pool flow. Here, numerical results demonstrate that the v P -v T Schur complement preconditioned solver scales well both algorithmically and in parallel, and is robust for highly ill-conditioned systems, for all tested rDG discretization schemes (up to 4th-order).

97 MATHEMATICS AND COMPUTING↗

Application of the multigrid method to grid generation

The multigrid method (MGM), used to numerically solve the pair of nonlinear elliptic equations commonly used to generate two dimensional boundary-fitted coordinate systems is discussed. Two different geometries are considered: one involving a coordinate system fitted about a circle and the other selected for an impinging jet flow problem. Two different relaxation schemes are tried: one is successive point overrelaxation and the other is a four-color scheme vectorizeable to take advantage of a parallel processor computer for greater computational speed. Results using MGM are compared with those using SOR (doing successive overrelaxations with the corresponding relaxation scheme on the fine grid only). It is found that MGM becomes significantly more effective than SOR as more accuracy is demanded and as more corrective grids, or more grid points, are used. For the accuracy required, it is found that MGM is two to three times faster than SOR in computing time. With the four-color relaxation scheme as applied to the impinging jet problem, the advantage of MGM over SOR is not as great. This may be due to the effect of a poor initial guess on MGM for this problem.

Ohring, S.↗

A Block-Structured Adaptive Mesh Framework to Solve Radiation Transfer Equation in Irregular Embedded Geometries

Radiation transport arises in various scientific, industrial, and medical fields, and understanding its effect in applications is needed to make accurate predictions, safety assessments and performance optimizations. Solving the Radiation Transport Equation (RTE) is challenging due to its integro-differential nature, which involves both differential and integral terms. The differential term describes the change in radiation intensity due to absorption and emission, while the integral term accounts for scattering. The accurate modeling of radiation is further complicated in many applications due to the complex, irregular geometries. Various methods exist for solving the RTE, including the zonal, Monte Carlo, spherical harmonics, discrete ordinates, and finite volume methods. Traditional mesh-based approaches, which rely on structured or unstructured meshes, struggle with irregular geometries due to: a) the difficulty of conforming structured grids to irregular domains, b) challenges in enforcing boundary conditions correctly, and c) the additional computational cost of unstructured mesh methods. This work presents a second-order accurate method for solving the RTE in irregular geometries. The radiation intensity is discretized using the finite-volume method in both spatial and angular directions on regular Cartesian grid blocks. Leveraging the block-structured adaptive mesh refinement (AMR) framework provided by AMReX, our method refines the grid locally to reduce spatial discretization error, ensuring a converged numerical solution while minimizing computational costs elsewhere. A two-stage deferred correction approach is employed: First, a first-order discretization on grid blocks is solved using an algebraic multigrid method in HYPRE. Second, a correction term is applied explicitly to achieve second-order accuracy. The correction term is calculated by approximating the radiation flux on cell faces using a Total Variation Diminishing (TVD) scheme. This approach ensures quick convergence of the multigrid method while preserving higher-order accuracy of the numerical solution. Irregular geometries are resolved as embedded boundaries (EB), resulting in both cut cells and regular cells. In cut cells, we modify the fluxes using face fractions and incorporate additional contributions from EB boundary conditions. To ensure higher-order convergence near the EB interface, the correction term is modified by interpolating the radiation intensity to fictitious ghost points. The implementation takes advantage of modern supercomputers by leveraging AMReX’sMPI/X parallelization strategy where X can be MPI or a GPU accelerator including CUDA, HIP and DPC++. We validate our solver using classical test cases, both with and without EB, demonstrating accuracy and efficiency. Additionally, we analyze the impact of adaptive mesh refinement on solution accuracy and computational cost, highlighting the advantages of our approach for high-resolution radiation transport simulations.

computational fluid dynamics (CFD)↗

Multigrid with Overlapping Patches

Solving boundary value problems with optimal efficiency requires adaptivity and multilevel techniques. Previously, an implementation of the AFACx algorithm is presented that is based on rectangular Cartesian grids. This implementation does not allow for the over]ap of grids that lie on the same level of refinement. We investigate the case in which these grids overlap. A standard technique for overlapping grids is the Schwarz algorithm. Some ways of using the Schwarz algorithm in a standard multigrid scheme are presented. Also, a problem that arises in some situations with non-aligned, overlapping grids is described. This situation comes up in a natural way when the Schwarz algorithm is used as a relaxation scheme within a multilevel algorithm. We identify the reason for the bad convergence and show that by more sophisticated interpolation the difficulties can be overcome. Then we present a multiplicative Schwarz algorithm for a large number of grids that has a high potential for parallelization. Finally we give some numerical results for the FACx algorithm with overlapping grids on each refinement level. The implementation of the described codes uses C++ and the array class libraries A++ and P++. Using the A++/P++ programming environment, it was possible to move from a serial code to a parallel code within a few days.

Berndt, Markus↗