Search NASA⌕ Search

SEARCH · Search NASA

Results for “multigrid”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40

Reaching bandwidth saturation using transparent injection parallelization

Although logically available, applications may not exploit enough instantaneous communication concurrency to maximize network utilization on HPC systems. This is exacerbated in hybrid programming models that combine single program multiple data with OpenMP or CUDA. We present the design of a “multi-threaded” runtime able to transparently increase the instantaneous network concurrency and to provide near saturation bandwidth, independent of the application configuration and dynamic behavior. The runtime offloads communication requests from application level tasks to multiple communication servers. The servers use system specific performance models to attain network saturation. Our techniques alleviate the need for spatial and temporal application level message concurrency optimizations. Experimental results show improved message throughput and bandwidth by as much as 150% for 4 KB messages on InfiniBand and by as much as 120% for 4 KB messages on Cray Aries. For more complex operations such as all-to-all collectives, we observe as much as 30% speedup. This translates into 23% speedup on 12,288 cores for a NAS FT implemented using FFTW. We observe as much as 76% speedup on 1500 cores for an already optimized UPC+OpenMP geometric multigrid application using hybrid parallelism. For the geometric multigrid GPU implementation, we observe as much as 44% speedup on 512 GPUs.

Chaimov, Nicholas↗

Numerical Methods for Large-Scale Data Factorization (Final Report (Milestone Date 9/30/11) for Subcontract No. B594099)

The following work has been performed by PI Hans De Sterck and graduate student Manda Winlaw for the required tasks 1-5 (as listed in the Statement of Work). Graduate student Manda Winlaw has visited LLNL January 31-March 11, 2011 and May 23-August 19, 2010, working with Van Henson and Mike O'Hara on non-negative matrix factorizations (NMF). She has investigated the dense subgraph clustering algorithm from 'Finding Dense Subgraphs for Sparse Undirected, Directed, and Bipartite Graphs' by Chen and Saad, testing this method on several term-document matrices and adapting it to cluster based on the rank of the subgraphs instead of the density. Manda Winlaw was awarded a first prize in the annual LLNL summer student poster competition for a poster on her NMF research. PI Hans De Sterck has developed a new adaptive algebraic multigrid algorithm for computing a few dominant or minimal singular triplets of sparse rectangular matrices. This work builds on adaptive algebraic multigrid methods that were further developed by the PI and collaborators (including Sanders and Henson) for Markov chains. The method also combines and extends existing multigrid algorithms for the symmetric eigenproblem. The PI has visited LLNL February 22-25, 2011, and has given a CASC seminar 'Algebraic Multigrid for the Singular Value Problem' on this work on February 23, 2011. During his visit, he has discussed this work and related topics with Van Henson, Geoffrey Sanders, Panayot Vassilevski, and others. He has tested the algorithm on PDE matrices and on a term-document matrix, with promising initial results. Manda Winlaw has also started to work, with O'Hara, on estimating probability distributions over undirected graph edges. The goal is to estimate probabilistic models from sets of undirected graph edges for the purpose of prediction, anomaly detection and support to supervised learning. Graduate student Manda Winlaw is writing a paper on the results obtained with O'Hara which will be submitted some time later in 2011 to a data mining conference. PI Hans De Sterck has developed a new optimization algorithm for canonical tensor approximation, formulating an extension of the nonlinear GMRES method to optimization problems. Numerical results for tensors with up to 8 modes show that this new method is efficient for sparse and dense tensors. He has written a paper on this which has been submitted to the SIAM Journal on Scientific Computing. PI Hans De Sterck has further developed his new optimization algorithm for canonical tensor approximation, formulating an extension in terms of steepest-descent preconditioning, which makes the approach generally applicable for nonlinear optimization. He has written a paper on this extension which has been submitted to Numerical Linear Algebra with Applications.

97 MATHEMATICS AND COMPUTING↗

The MueLu Tutorial

The MueLu tutorial is written as a hands-on tutorial for MueLu, the next generation multigrid framework in Trilinos. It covers the whole spectrum from absolute beginners’ topics to expert level. Since the focus of this tutorial is on practical and technical aspects of multigrid methods in general and MueLu in particular, the reader is expected to have a basic understanding of multigrid methods and its general underlying concepts. Please refer to multigrid textbooks (e.g. [1]) for the theoretical background.

99 GENERAL AND MISCELLANEOUS↗

Computational techniques for high-speed flows with viscous and chemical effects

Algorithms for solving the Euler and the Navier-Stokes equations in conjunction with chemical kinetic equations are presented. The convective flux is estimated from a quasi one dimensional interpolation procedure. Shock, contact, and expansion waves and thermochemical nonequilibrium phenomena are captured by the Lax-Friedrichs technique. Relaxation techniques were developed to enhance their effectiveness in dealing with spatial and temporal stiffness associated with the physical problems. Both explicit and implicit smoothers were implemented into the standard multigrid time stepping method. Unsteady and steady scalar problems are discussed. A perfect gas and equilibrium air shock tube problem is investigated. Numerical schemes and techniques are compared for the problems of shock and boundary layer interaction and three dimensional viscous, nonequilibrium flow encompassing an aerobrake. The results are comparable in accuracy against other high order non-oscillatory techniques. The multigrid methods are assessed using a Mach 8 flow over a complete planar body, a sphere, and a blunt delta wing at 20 deg incidence. Applying an implicit multigrid method on a nested grid of 128 by 64 nodes the reduction factor is 0.25. The central processing unit reduction factor is 2.2 after both the single and multigrid Runge-Kutta solutions converged to machine zero on a grid of 37 by 41 by 73 nodes.

Li, C. P.↗

On Bi-Grid Local Mode Analysis of Solution Techniques for 3-D Euler and Navier-Stokes Equations

A procedure is presented for utilizing a bi-grid stability analysis as a practical tool for predicting multigrid performance in a range of numerical methods for solving Euler and Navier-Stokes equations. Model problems based on the convection, diffusion and Burger's equation are used to illustrate the superiority of the bi-grid analysis as a predictive tool for multigrid performance in comparison to the smoothing factor derived from conventional von Neumann analysis. For the Euler equations, bi-grid analysis is presented for three upwind difference based factorizations, namely Spatial, Eigenvalue and Combination splits, and two central difference based factorizations, namely LU and ADI methods. In the former, both the Steger-Warming and van Leer flux-vector splitting methods are considered. For the Navier-Stokes equations, only the Beam-Warming (ADI) central difference scheme is considered. In each case, estimates of multigrid convergence rates from the bi-grid analysis are compared to smoothing factors obtained from single-grid stability analysis. Effects of grid aspect ratio and flow skewness are examined. Both predictions are compared with practical multigrid convergence rates for 2-D Euler and Navier-Stokes solutions based on the Beam-Warming central scheme.

Ibraheem, S. O.↗

On DGS Relaxation: The Stokes Problem

Multigrid methods have proven to be efficient methods for solving partial differential equations (especially those of elliptic type). There is also growing experience with multigrid solvers for fluids problems, e.g., the Stokes and Navier-Stokes equations (using both finite element and finite difference discretizations). It is also well known that at the heart of any multigrid method is the smoother. In this work we look at a smoother introduced by Brandt and Dinar (DGS relaxation), and we examine some of its properties and consider some possible modifications to it. It is well known that multigrid performance using DGS relaxation is sensitive to the treatment of boundaries; this issue is addressed.

Meir, A. J.↗

Effective Boundary Treatment for the Biharmonic Dirichlet Problem

The biharmonic equation can be rewritten as a system of two Poisson equations. Multigrid solution of this system is expected to converge with the same amount of work as solving two Poisson equations, requiring less than 70 floating point operations (scalar multiply or addition) per fine grid point to reach a solution using an FMG algorithm. For periodic boundary conditions, this goal is attained by simple, straightforward application of multigrid. For Dirichlet boundary conditions, however, convergence is impeded by poor interaction with the boundaries. Attempts to overcome the slowness without specifically addressing the boundaries have resulted in multigrid algorithms not attaining the Poisson convergence rate. We present three methods of boundary treatment with which full multigrid efficiency can be obtained. All implement an approach described by Brandt, concentrating some additional effort near the boundary. The first approach simply adds a number of relaxation sweeps over points close to the boundary. The second uses joint relaxation on near-boundary points. The third method takes something from each of the first two methods, resulting in a solver more suitable for highly parallel applications.

Brandt, A.↗

Spatially adaptive stochastic methods for fluid–structure interactions subject to thermal fluctuations in domains with complex geometries

We develop stochastic mixed finite element methods for spatially adaptive simulations of fluid–structure interactions when subject to thermal fluctuations. To account for thermal fluctuations, we introduce a discrete fluctuation–dissipation balance condition to develop compatible stochastic driving fields for our discretization. We perform analysis that shows our condition is sufficient to ensure results consistent with statistical mechanics. We show the Gibbs–Boltzmann distribution is invariant under the stochastic dynamics of the semi-discretization. To generate efficiently the required stochastic driving fields, we develop a Gibbs sampler based on iterative methods and multigrid to generate fields with O(N) computational complexity. Our stochastic methods provide an alternative to uniform discretizations on periodic domains that rely on Fast Fourier Transforms. To demonstrate in practice our stochastic computational methods, we investigate within channel geometries having internal obstacles and no-slip walls how the mobility/diffusivity of particles depends on location. Our methods extend the applicability of fluctuating hydrodynamic approaches by allowing for spatially adaptive resolution of the mechanics and for domains that have complex geometries relevant in many applications.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Massively parallel solvers for elliptic partial differential equations in numerical weather and climate prediction

The demand for substantial increases in the spatial resolution of global weather and climate prediction models makes it essential to use numerically efficient and highly scalable algorithms to solve the equations of large-scale atmospheric fluid dynamics. For stability and efficiency reasons, several of the operational forecasting centres, in particular the Met Office and the European Centre for Medium-Range Weather Forecasts (ECMWF) in the UK, use semi-implicit semi-Lagrangian time-stepping in the dynamical core of the model. The additional burden with this method is that a three-dimensional elliptic partial differential equation (PDE) for the pressure correction has to be solved at every model time step and this often constitutes a significant proportion of the time spent in the dynamical core. In global models, this PDE must be solved in a thin spherical shell. To run within tight operational time-scales, the solver has to be parallelized and there seems to be a (perceived) misconception that elliptic solvers do not scale to large processor counts and hence implicit time-stepping cannot be used in very high-resolution global models. After reviewing several methods for solving the elliptic PDE for the pressure correction and their application in atmospheric models, we demonstrate the performance and very good scalability of Krylov subspace solvers and multigrid algorithms for a representative model equation with more than 1010 unknowns on 65 536 cores on the High-End Computing Terascale Resource (HECToR), the UK's national supercomputer. Here, we tested and optimized solvers from two existing numerical libraries (the Distributed and Unified Numerics Environment (DUNE) and Parallel High Performance Preconditioners (hypre)) and implemented both a conjugate gradient solver and a geometric multigrid algorithm based on a tensor-product approach, which exploits the strong vertical anisotropy of the discretized equation. We study both weak and strong scalability and compare the absolute solution times for all methods; in contrast to one-level methods, the multigrid solver is robust with respect to parameter variations.

58 GEOSCIENCES↗

A non-intrusive parallel-in-time adjoint solver with the XBraid library

In this paper, an adjoint solver for the multigrid-in-time software library XBraid is presented. XBraid provides a non-intrusive approach for simulating unsteady dynamics on multiple processors while parallelizing not only in space but also in the time domain (XBraid: Parallel multigrid in time, http://llnl.gov/casc/xbraid). It applies an iterative multigrid reduction in time algorithm to existing spatially parallel classical time propagators and computes the unsteady solution parallel in time. Techniques from Automatic Differentiation are used to develop a consistent discrete adjoint solver which provides sensitivity information of output quantities with respect to design parameter changes. The adjoint code runs backwards through the primal XBraid actions and accumulates gradient information parallel in time. It is highly non-intrusive as existing adjoint time propagators can easily be integrated through the adjoint interface. The adjoint code is validated on advection-dominated flow with periodic upstream boundary condition. We report it provides similar strong scaling results as the primal XBraid solver and offers great potential for speeding up the overall computational costs for sensitivity analysis using multiple processors.

97 MATHEMATICS AND COMPUTING↗

A space-time parallel algorithm with adaptive mesh refinement for computational fluid dynamics

This work describes a space-time parallel algorithm with space-time adaptive mesh refinement (AMR). AMR with subcycling is added to multigrid reduction-in-time (MGRIT) in order to provide solution efficient adaptive grids with a reduction in work performed on coarser grids. This algorithm is achieved by integrating two software libraries: XBraid (Parallel time integration with multigrid. https://computation.llnl.gov/projects/parallel-timeintegration-multigrid) and Chombo (Chombo software package for AMR applications—design document, 2014). The former is a parallel time integration library using multigrid and the latter is a massively parallel structured AMR library. Employing this adaptive space-time parallel algorithm is Chord (Comput Fluids 123:202–217, 2015), a computational fluid dynamics (CFD) application code for solving compressible fluid dynamics problems. For the same solution accuracy, speedups are demonstrated from the use of space-time parallelization over the time-sequential integration on Couette flow and Stokes’ second problem. On a transient Couette flow case, at least a 1.5× speedup is achieved, and with a time periodic problem, a speedup of up to 13.7× over the time-sequential case is obtained. In both cases, the speedup is achieved by adding processors and exploring additional parallelization in time. The numerical experiments show the algorithm is promising for CFD applications that can take advantage of the time parallelism. Future work will focus on improving the parallel performance and providing more tests with complex fluid dynamics to demonstrate the full potential of the algorithm.

97 MATHEMATICS AND COMPUTING↗

Efficient discontinuous Galerkin implementations and preconditioners for implicit unsteady compressible flow simulations

This work presents and compares efficient implementations of high-order discontinuous Galerkin methods: a modal matrix-free discontinuous Galerkin (DG) method, a hybridizable discontinuous Galerkin (HDG) method, and a primal formulation of HDG, applied to the implicit solution of unsteady compressible flows. The matrix-free implementation allows for a reduction of the memory footprint of the solver when dealing with implicit time-accurate discretizations. HDG reduces the number of globally-coupled degrees of freedom relative to DG, at high order, by statically condensing element-interior degrees of freedom from the system in favor of face unknowns. The primal formulation further reduces the element-interior degrees of freedom by eliminating the gradient as a separate unknown. This paper introduces a p-multigrid preconditioner implementation for these discretizations and presents results for various flow problems. Benefits of the p-multigrid strategy relative to simpler, less expensive, preconditioners are observed for stiff systems, such as those arising from low-Mach number flows at high-order approximation. The p-multigrid preconditioner also shows excellent scalability for parallel computations. Finally, additional savings in both speed and memory occur with a matrix-free/reduced version of the preconditioner.

97 MATHEMATICS AND COMPUTING↗

Compiler-Directed Transformation for Higher-Order Stencils

As the cost of data movement increasingly dominates performance, developers of finite-volume and finite-difference solutions for partial differential equations (PDEs) are exploring novel higher-order stencils that increase numerical accuracy and computational intensity. This paper describes a new compiler reordering transformation applied to stencil operators that performs partial sums in buffers, and reuses the partial sums in computing multiple results. This optimization has multiple effect son improving stencil performance that are particularly important to higher-order stencils: exploits data reuse, reduces floating-point operations, and exposes efficient SIMD parallelism to backend compilers. We study the benefit of this optimization in the context of Geometric Multigrid (GMG), a widely used method to solvePDEs, using four different Jacobi smoothers built from 7-, 13-, 27-and 125-point stencils. We quantify performance, speedup, andnumerical accuracy, and use the Roofline model to qualify our results. Ultimately, we obtain over 4× speedup on the smoothers themselves and up to a 3× speedup on the multigrid solver. Finally, we demonstrate that high-order multigrid solvers have the potential of reducing total data movement and energy by several orders of magnitude.

97 MATHEMATICS AND COMPUTING↗

Optimal-Order Preconditioners for Linear Systems Arising in the Semismooth Newton Solution of a Class of Control-Constrained Problems

We present a new multigrid preconditioner for the linear systems arising in the semismooth Newton method solution of certain control-constrained, quadratic distributed optimal control problems. Using a piecewise constant discretization of the control space, each semismooth Newton iteration essentially requires inverting a principal submatrix of the matrix entering the normal equations of the associated unconstrained optimal control problem, the rows (and columns) of the submatrix representing the constraints deemed inactive at the current iteration. Previously developed multigrid preconditioners for the aforementioned submatrices were based on constructing a sequence of conforming coarser spaces and proved to be of suboptimal quality for the class of problems considered. Instead, the multigrid preconditioner introduced in this work uses nonconforming coarse spaces, and it is shown that, under reasonable geometric assumptions on the constraints that are deemed inactive, the preconditioner approximates the inverse of the desired submatrix to optimal order. The preconditioner is tested numerically on a classical elliptic-constrained optimal control problem and further on a constrained image-deblurring problem.

97 MATHEMATICS AND COMPUTING↗

αAMG based on Weighted Matching for Systems of Elliptic PDEs Arising From Displacement and Mixed Methods

Adaptive Algebraic Multigrid (or Multilevel) Methods (αAMG) are introduced to improve robustness and efficiency of classical algebraic multigrid methods in dealing with problems where no a-priori knowledge or assumptions on the near-null kernel of the underlined matrix are available. Recently we proposed an adaptive (bootstrap) AMG method, αAMG, aimed to obtain a composite solver with a desired convergence rate. Each new multigrid component relies on a current (general) smooth vector and exploits pairwise aggregation based on weighted matching in a matrix graph to define a new automatic, general-purpose coarsening process, which we refer to as “the compatible weighted matching”. In this work, we present results that broaden the applicability of our method to different finite element discretizations of elliptic PDEs. In particular, we consider systems arising from displacement methods in linear elasticity problems and saddle-point systems that appear in the application of the mixed method to Darcy problems.

97 MATHEMATICS AND COMPUTING↗

Array-based, parallel hierarchical mesh refinement algorithms for unstructured meshes

In this paper, we describe an array-based hierarchical mesh refinement capability through uniform refinement of unstructured meshes for efficient solution of PDE's using finite element methods and multigrid solvers. A multi-degree, multi-dimensional and multi-level framework is designed to generate the nested hierarchies from an initial coarse mesh that can be used for a variety of purposes such as in multigrid solvers/preconditioners, to do solution convergence and verification studies and to improve overall parallel efficiency by decreasing I/O bandwidth requirements (by loading smaller meshes and in memory refinement). We also describe a high-order boundary reconstruction capability that can be used to project the new points after refinement using high-order approximations instead of linear projection in order to minimize and provide more control on geometrical errors introduced by curved boundaries.The capability is developed under the parallel unstructured mesh framework "Mesh Oriented dAtaBase" (MOAB Tautges et al. (2004)). We describe the underlying data structures and algorithms to generate such hierarchies in parallel and present numerical results for computational efficiency and effect on mesh quality. Furthermore, we also present results to demonstrate the applicability of the developed capability to study convergence properties of different point projection schemes for various mesh hierarchies and to a multigrid finite-element solver for elliptic problems.

97 MATHEMATICS AND COMPUTING↗

Petascale solvers for anisotropic PDEs in atmospheric modelling on GPU clusters

Memory bound applications such as solvers for large sparse systems of equations remain a challenge for GPUs. Fast solvers should be based on numerically efficient algorithms and implemented such that global memory access is minimised. To solve systems with trillions ( O ( 10 12 ) ) unknowns the code has to make efficient use of several million individual processor cores on large GPU clusters. We describe the multi-GPU implementation of two algorithmically optimal iterative solvers for anisotropic PDEs which are encountered in (semi-) implicit time stepping procedures in atmospheric modelling. In this application the condition number is large but independent of the grid resolution and both methods are asymptotically optimal, albeit with different absolute performance. In particular, an important constant in the discretisation is the CFL number; only the multigrid solver is robust to changes in this constant. We parallelise the solvers and adapt them to the specific features of GPU architectures, paying particular attention to efficient global memory access. We achieve a performance of up to 0.78 PFLOPs when solving an equation with 0.55 · 10 12 unknowns on 16384 GPUs; this corresponds to about 3% of the theoretical peak performance of the machine and we use more than 40% of the peak memory bandwidth with a Conjugate Gradient (CG) solver. Although the other solver, a geometric multigrid algorithm, has a slightly worse performance in terms of FLOPs per second, overall it is faster as it needs less iterations to converge; the multigrid algorithm can solve a linear PDE with half a trillion unknowns in about one second.

Computer Science↗

Topology-Aware Performance Optimization and Modeling of Adaptive Mesh Refinement Codes for Exascale

Here, we introduce a topology-aware performance optimization and modeling workflow for AMR simulation that includes two new modeling tools, ProgrAMR and Mota Mapper, which interface with the BoxLib AMR framework and the SSTmacro network simulator. ProgrAMR allows us to generate and model the execution of task dependency graphs from high-level specifications of AMR-based applications, which we demonstrate by analyzing two example AMR-based multigrid solvers with varying degrees of asynchrony. Mota Mapper generates multiobjective, network topology-aware box mappings, which we apply to optimize the data layout for the example multigrid solvers. While the sensitivity of these solvers to layout and execution strategy appears to be modest for balanced scenarios, the impact of better mapping algorithms can be significant when performance is highly constrained by network hop latency. Furthermore, we show that network latency in the multigrid bottom solve is the main contributing factor preventing good scaling on exascale-class machines.

97 MATHEMATICS AND COMPUTING↗