Search NASA⌕ Search

SEARCH · Search NASA

Results for “multigrid”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49

Implementation of a Mesh refinement algorithm into the quasi-static PIC code QuickPIC

Plasma-based acceleration (PBA) has emerged as a promising candidate for the accelerator technology used to build a future linear collider and/or an advanced light source. In PBA, a trailing or witness particle beam is accelerated in the plasma wave wakefield (WF) created by a laser or particle beam driver. The WF is often nonlinear and involves the crossing of plasma particle trajectories in real space and thus particle-in-cell methods are used. The distance over which the drive beam evolves is several orders of magnitude larger than the wake wavelength. This large disparity in length scales is amenable to the quasi-static approach. Three-dimensional (3D), quasi-static (QS), particle-in-cell (PIC) codes, e.g., QuickPIC, have been shown to provide high fidelity simulation capability with 2-4 orders of magnitude speedup over 3D fully explicit PIC codes. In PBA, the witness beam needs to be matched to the focusing forces of the WF to reduce the emittance growth. In some linear collider designs, the matched spot size of the witness beam can be 2 to 3 orders of magnitude smaller than the spot size (and wavelength) of the wakefield. Such an additional disparity in length scales is ideal for mesh refinement where the WF within the witness beam is described on a finer mesh than the rest of the WF. A mesh refinement scheme is described that has been implemented into the 3D QS PIC code, QuickPIC. Very fine (high) resolution is used in a small spatial region that includes the witness beam and progressively coarser resolutions in the rest of the simulation domain. A fast multigrid Poisson solver has been implemented for the field solve on the refined meshes and a Fast Fourier Transform (FFT) based Poisson solver is used for the coarse mesh. The code has been parallelized with both MPI and OpenMP, and the parallel scalability has also been improved by using pipelining. A preliminary adaptive mesh refinement technique is described to optimize the computational time for simulations with an evolving witness beam size. Several test problems are used to verify that the mesh refinement algorithm provides accurate results. Additionally, the results are benchmarked against highly resolved simulations exhibiting near-azimuthal symmetry, performed using QPAD—a novel hybrid QS PIC code that uses a PIC description in the coordinates (r, ct – z) and a gridless description in the azimuthal angle, Φ.

Linear collider↗

Reinforcing of QA/QC programs in radiotherapy departments in Croatia: Results of treatment planning system verification

Implementation of advanced techniques in clinical practice can greatly improve the outcome of radiation therapy, but it also makes the process much more complex with a lot of room for errors. An important part of the quality assurance program is verification of treatment planning system (TPS). Dosimetric verifications in anthropomorphic phantom were performed in 4 centers where new systems were installed. A total of 14 tests for 2 photon energies and multigrid superposition algorithms were conducted using the CMS XiO TPS. Evaluation criteria as specified in the International Atomic Energy Agency Technical Reports Series (IAEA TRS) 430 were employed. Results of measurements are grouped according to the placement of the measuring point and the beam energy. The majority of differences between calculated and measured doses in the water-equivalent part of the phantom were in tolerance. Significantly more out-of-tolerance values were observed in “nonwater-equivalent” parts of the phantom, especially for higher-energy photon beams. This survey was done as a part of continuous effort to build up awareness of quality assurance/quality control (QA/QC) importance in the Croatian radiotherapy community. Understanding the limitations of different parts of the various systems used in radiation therapy can systematically improve quality as well.

61 RADIATION PROTECTION AND DOSIMETRY↗

Multilevel acceleration of scattering-source iterations with application to electron transport

Acceleration/preconditioning strategies available in the SCEPTRE radiation transport code are described. A flexible transport synthetic acceleration (TSA) algorithm that uses a low-order discrete-ordinates (S N ) or spherical-harmonics (P N ) solve to accelerate convergence of a high-order S N source-iteration (SI) solve is described. Convergence of the low-order solves can be further accelerated by applying off-the-shelf incomplete-factorization or algebraic-multigrid methods. Also available is an algorithm that uses a generalized minimum residual (GMRES) iterative method rather than SI for convergence, using a parallel sweep-based solver to build up a Krylov subspace. TSA has been applied as a preconditioner to accelerate the convergence of the GMRES iterations. The methods are applied to several problems involving electron transport and problems with artificial cross sections with large scattering ratios. These methods were compared and evaluated by considering material discontinuities and scattering anisotropy. Observed accelerations obtained are highly problem dependent, but speedup factors around 10 have been observed in typical applications.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Hierarchical Krylov and nested Krylov methods for extreme-scale computing

The solution of large, sparse linear systems is typically a dominant phase of computation for simulations based on partial differential equations, which are ubiquitous in scientific and engineering applications. While preconditioned Krylov methods are widely used and provide many advantages for solving sparse linear systems that do not have highly convergent, geometric multigrid solvers or specialized fast solvers, Krylov methods encounter well-known scaling difficulties for over 10,000 processor cores because each iteration requires at least one vector inner product, which in turn requires a global synchronization that scales poorly because of internode latency. To aid in overcoming these difficulties, we have developed hierarchical Krylov methods and nested Krylov methods in the PETSc library that reduce the number of global inner products required across the entire system (where they are expensive), though freely allow vector inner products across smaller subsets of the entire system (where they are inexpensive) or use inner iterations that do not invoke vector inner products at all. Nested Krylov methods are a generalization of inner-outer iterative methods with two or more layers. Hierarchical Krylov methods are a generalization of block Jacobi and overlapping additive Schwarz methods, where each block itself is solved by Krylov methods on smaller blocks. Conceptually, the hierarchy can continue recursively to an arbitrary number of levels of smaller and smaller blocks. As a specific case, we introduce the hierarchical FGMRES method, or h-FGMRES, and we demonstrate the impact of two-level h-FGMRES with a variable preconditioner on the PFLOTRAN subsurface flow application. We also demonstrate the impact of nested FGMRES, BiCGStab and Chebyshev methods. These hierarchical Krylov methods and nested Krylov methods significantly reduced overall PFLOTRAN simulation time on the Cray XK6 when using 10,000 through 224,000 cores through the combined effects of reduced global synchronization due to fewer global inner products and stronger inner hierarchical or nested preconditioners.

97 MATHEMATICS AND COMPUTING↗

Tausch: A halo exchange library for large heterogeneous computing systems using MPI, OpenCL, and CUDA

Exchanging halo data is a common task in modern scientific computing applications and efficient handling of this operation is critical for the performance of the overall simulation. Tausch is a novel header-only library that provides a simple API for efficiently handling these types of data movements. Tausch supports both simple CPU-only systems, but also more complex heterogeneous systems with both CPUs and GPUs. It currently supports both OpenCL and CUDA for communicating with GPGPU devices, and allows for communication between GPGPUs and CPUs. The API allows for drop-in replacement in existing codes and can be used for the communication layer in new codes. This paper provides an overview of the approach taken in Tausch, and a performance analysis that demonstrates expected and achieved performance. Here, we highlight the ease of use and performance with three applications: First Tausch is compared to the halo exchange framework from two Mantevo applications, HPCCG and miniFE, and then it is used to replace a legacy halo exchange library in the flexible multigrid solver framework Cedar.

97 MATHEMATICS AND COMPUTING↗

A Comparison of Linear Solvers for Resolving Flow in Three-Dimensional Discrete Fracture Networks

We compare various methods for resolving steady flow within three-dimensional discrete fracture networks, including direct methods, Krylov subspace methods with and without preconditioning, and multi-grid methods. We compared the performance of the methods based on compute times and scaling of the solution as a function of the number of grid nodes and log-variance of the hydraulic aperture. The methods are applied to three test cases: (a) variable density of networks with a truncated power-law distribution of fracture lengths, (b) a fixed network composed of monodisperse fracture sizes but varied permeability/aperture heterogeneity, (c) and a network based on field site in Nevada, US. We chose these cases to allow us to study the impact of the mesh size and flow properties, as well as to demonstrate our conclusions on a large-scale, realistic problem (more than 40 million mesh nodes). A direct solution using Cholesky factorization outperformed other methods for every example but was closely followed in performance by some algebraic multigrid (AMG) preconditioned Krylov subspace methods. Among the Krylov methods, conjugate gradients (CG) with an AMG preconditioner performs the best. Generally, Cholesky factorization is recommended, but CG with an AMG preconditioner may be suitable for very large problems beyond 40 million nodes where the entire linear system cannot reside in memory.

58 GEOSCIENCES↗

Origin of large variations of current on/off ratio and switching voltage in atomically thin memristors: an exascale ab initio transport study

Nonvolatile resistive switching in two-dimensional monolayers opens a new avenue for high-density memory/computing devices. However, questions remain as to why the current on/off ratio and switching voltage vary significantly among different devices. Here, we simulate electronic transport of large systems consisting of a h-BN monolayer sandwiched by gold electrodes, enabled by an implementation of the nonequilibrium Green’s function method in the exascale density functional theory (DFT) code: Real-space MultiGrid. Systematic calculations reveal that the wide range of on/off ratios is due to variations in interface distances between the electrode and h-BN that significantly modulate their wavefunction overlap. In addition, DFT calculations demonstrate that the energy barrier of a gold atom dissociating from the electrode to h-BN increases dramatically with the interface distance, thereby explaining the strong dependence of the switching voltage on distance. Our work demonstrates the significance of interface distance in governing the current on/off ratio and switching voltage.

Electronic devices↗

Quantum effects on the dynamics and properties of soft materials

The quantum effects of nuclear and electronic motion play an important role in the structure, dynamics, and function of soft materials, yet they are difficult to capture with conventional classical simulations or static electronic–structure methods. In this work several complementary approaches for treating quantum effects in polymeric and soft–matter systems are demonstrated, with a focus being on the hydrogen-bonded networks, ion and charge transport, and photoactive chromophores. The proton transfer, tunneling, and isotope effects are captured within the reduced-dimensionality models by implementing grid-based nuclear quantum dynamics in terms of the discrete variable and Fourier bases. The nuclear quantum dynamics is extended to larger systems by employing the quantum trajectories and quantum–thermal bath schemes combined with on-the-fly electronic structure, enabling the description of high-dimensional polymeric environments at feasible cost. The dynamics in the electronic degrees of freedom, simulating the optical response in large chromophores such as chlorophylls, is performed using the real-time time-dependent density functional theory implemented in the real-space multigrid (RMG) code. These approaches are demonstrated on case studies of the proton and hydroxide transport in hydrated polymer membranes, charge transfer in conjugated polymers, and the optical spectra of chlorophyll chromophores relevant to polymerized chlorophyll materials and chlorophyll–polymer hybrids. The reviewed methods and applications highlight practical routes of including quantum effects in simulations of soft functional materials.

Garashchuk, Sophya [Univ. of South Carolina, Colum↗

Analyses of high power negative ion accelerators for ITER neutral beam injector (invited)

In JAEA, research and developments to realize high power accelerator (1 MeV, 40 AD{sup -} ion beams for 3600 s) for ITER have been carried out experimentally and numerically utilizing a five stage MAMuG (Multiaperture, Multigrid) accelerator. In this paper, the extension of the gap length, which is required to improve the voltage holding capability, is examined in two dimensional beam optics analyses and also from view point of stripping loss of ions. In order to suppress excess power loadings due to the direct interception of negative ions, which is issued in long pulse tests, the beamlet deflection is analyzed in three dimensional multibeamlet analyses. The necessary modifications shown above are applied to the MAMuG accelerator for coming long pulse tests in JAEA and ITER.

43 PARTICLE ACCELERATORS↗

Long pulse H{sup -} ion beam acceleration in MeV accelerator

A multiaperture multigrid accelerator called ''MeV accelerator'' has been developed for neutral beam injection system of international thermonuclear experimental reactor. In the present work, long pulse H{sup -} ion beam acceleration was performed by the MeV accelerator equipped with new water-cooled grids. At present, the pulse length was extended to 5 s for the beams of 750 keV, 221 mA, and 10 s for the beams of 600 keV, 158 mA. Energy density, defined as products of beam energy (keV), current (mA), and pulse (s) divided by aperture area (m{sup 2}), increased more than one order of magnitude higher compared with original MeV accelerator without water cooling in its grids. At higher energy and current, the grid was melted by beam deflection. Due to this grid melting, breakdowns occurred between the grids, and hence, the pulse length was limited. Beam deflection will be compensated by aperture displacement in next experiment.

43 PARTICLE ACCELERATORS↗

A divide-conquer-recombine algorithmic paradigm for large spatiotemporal quantum molecular dynamics simulations

We introduce an extension of the divide-and-conquer (DC) algorithmic paradigm called divide-conquer-recombine (DCR) to perform large quantum molecular dynamics (QMD) simulations on massively parallel supercomputers, in which interatomic forces are computed quantum mechanically in the framework of density functional theory (DFT). In DCR, the DC phase constructs globally informed, overlapping local-domain solutions, which in the recombine phase are synthesized into a global solution encompassing large spatiotemporal scales. For the DC phase, we design a lean divide-and-conquer (LDC) DFT algorithm, which significantly reduces the prefactor of the O(N) computational cost for N electrons by applying a density-adaptive boundary condition at the peripheries of the DC domains. Our globally scalable and locally efficient solver is based on a hybrid real-reciprocal space approach that combines: (1) a highly scalable real-space multigrid to represent the global charge density; and (2) a numerically efficient plane-wave basis for local electronic wave functions and charge density within each domain. Hybrid space-band decomposition is used to implement the LDC-DFT algorithm on parallel computers. A benchmark test on an IBM Blue Gene/Q computer exhibits an isogranular parallel efficiency of 0.984 on 786 432 cores for a 50.3 × 10{sup 6}-atom SiC system. As a test of production runs, LDC-DFT-based QMD simulation involving 16 661 atoms is performed on the Blue Gene/Q to study on-demand production of hydrogen gas from water using LiAl alloy particles. As an example of the recombine phase, LDC-DFT electronic structures are used as a basis set to describe global photoexcitation dynamics with nonadiabatic QMD (NAQMD) and kinetic Monte Carlo (KMC) methods. The NAQMD simulations are based on the linear response time-dependent density functional theory to describe electronic excited states and a surface-hopping approach to describe transitions between the excited states. A series of techniques are employed for efficiently calculating the long-range exact exchange correction and excited-state forces. The NAQMD trajectories are analyzed to extract the rates of various excitonic processes, which are then used in KMC simulation to study the dynamics of the global exciton flow network. This has allowed the study of large-scale photoexcitation dynamics in 6400-atom amorphous molecular solid, reaching the experimental time scales.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Limitation of parallel flow in double diffusive convection: Two- and three-dimensional transitions in a horizontal porous domain

Two-dimensional (2D) and three-dimensional (3D) numerical simulations of double diffusion natural convection in an elongated enclosure filled with a binary fluid saturating a porous medium are carried out in the present work. The Boussinesq approximation is made in the formulation of the problem, and Neumann boundary conditions for temperature and concentration are adopted, respectively, on vertical and horizontal walls of the cavity. The used numerical method is based on the control volume approach, with the third order quadratic upstream interpolation scheme in approximating the advection terms. A semi implicit method algorithm is used to handle the velocity-pressure coupling. To avoid the excessively high computer time inherent to the solution of 3D natural convection problems, full approximation storage with full multigrid method is used to solve the problem. A wide range of the controlling parameters (Rayleigh-Darcy number Ra, lateral aspect ratio Ay, Lewis number Le, and the buoyancy ration N) is investigated. We clearly show that increasing the depth of the cavity (i.e., the lateral aspect ratio) has an important effect on the flow patterns. The 2D perfect parallel flows obtained for small lateral aspect ratio are drastically destabilized by increasing the cavity lateral dimension. This yields a 3D fluid motion with a much more complex flow pattern and the usually considered 2D parallel flow model cannot be applied.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Measuring ion velocity distribution functions through high-aspect ratio holes in inductively coupled plasmas

Several issues associated with plasma etching of high aspect ratio structures originate from the ions' bombardment of the sidewalls of the feature. The off normal angle incident ions are primarily due to their temperature at the sheath edge and possibly to charging effects. We have measured the ion velocity distribution function (IVDF) at the wafer surface in an industrial inductively coupled plasma reactor by using multigrid retarding field analyzers (RFA) in front of which we place 400 μm thick capillary plates with holes of 25, 50, and 100 μm diameters. The RFA then probes IVDF at the exit of the holes with Aspect Ratios (AR) of 16, 8, and 4, respectively. The results show that the ion flux dramatically drops with the increase in AR. By comparing the measured IVDF with an analytical model, we concluded that the ion temperature is 0.27 eV in our plasma conditions. The charging effects are also observed and are shown to significantly reduce the ion energy at the bottom of the feature but only with a “minor” effect on the ion flux and the shape of the IVDF.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Semi-implicit continuum kinetic modeling of weakly collisional parallel transport in a magnetic mirror

We present implicit-explicit (IMEX) kinetic simulations of weakly collisional parallel plasma transport in magnetic mirror configurations using the continuum code COGENT. The numerical scheme employs a Jacobian-free Newton–Krylov method with algebraic multigrid preconditioning to overcome the severe time step limitations imposed by strong mirror forces in fully explicit schemes. Applied to parameters relevant to the Wisconsin HTS Axisymmetric Mirror experiment, the IMEX approach enables time steps up to 2.5×10 4 times larger than those permitted by explicit methods, resulting in a 2500× speedup in 1D–2V simulations of parallel transport with kinetic ions and Boltzmann electrons. Additionally, a reduced bounce-averaged model for a square mirror is implemented to support the computationally intensive fully kinetic simulations. The bounce-averaged formulation is used to evaluate the numerical convergence of the velocity-space discretization algorithms and to assess the role of the collision model by comparing simulations employing the nonlinear Fokker–Planck and the simplified Lenard–Bernstein–Dougherty collision operators.

Collision theories↗

pyRMG: A framework for high-throughput, large-cell DFT calculations on supercomputers

Exascale computing delivers the raw power to simulate ever larger and more chemically realistic systems, but realizing this potential requires codes that can efficiently use thousands of processors. Our real-space multigrid (RMG) density functional theory (DFT) code’s grid-decomposition approach scales nearly linearly with the number of graphics processing units (GPUs), even for simulations exceeding thousands of atoms. This scalability makes RMG a compelling tool for high-throughput DFT studies of materials that would otherwise be bottlenecked in other codes (for example, by global fast Fourier transforms in plane-wave DFT). However, the limited workflow infrastructure for RMG has thus far constrained its adoption to a small user community. In this work, we present pyRMG, a Python package designed to streamline the setup and execution of RMG DFT calculations. Built on the pymatgen and ASE (Atomic Simulation Environment) computational materials science Python packages, pyRMG automates input generation and convergence checking, and it integrates with modern job schedulers (e.g., Flux) on leadership-class platforms such as Frontier and Perlmutter. Here, we demonstrate pyRMG for a high-throughput study of strain effects in 2D 2L-Bi 2 Se 3 /2L-NbSe 2 heterostructures, which offers chemical insights into this system and shows that RMG-based workflows can converge with limited user intervention.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Eigenvalue Solvers for Modeling Nuclear Reactors on Leadership Class Machines

In this paper, three complementary methods have been implemented in the code Denovo that accelerate neutral particle transport calculations with methods that use leadership-class computers fully and effectively: a multigroup block (MG) Krylov solver, a Rayleigh quotient iteration (RQI) eigenvalue solver, and a multigrid in energy (MGE) preconditioner. The MG Krylov solver converges more quickly than Gauss Seidel and enables energy decomposition such that Denovo can scale to hundreds of thousands of cores. RQI should converge in fewer iterations than power iteration (PI) for large and challenging problems. RQI creates shifted systems that would not be tractable without the MG Krylov solver. It also creates ill-conditioned matrices. The MGE preconditioner reduces iteration count significantly when used with RQI and takes advantage of the new energy decomposition such that it can scale efficiently. Each individual method has been described before, but this is the first time they have been demonstrated to work together effectively. The combination of solvers enables the RQI eigenvalue solver to work better than the other available solvers for large reactors problems on leadership-class machines. Using these methods together, RQI converged in fewer iterations and in less time than PI for a full pressurized water reactor core. These solvers also performed better than an Arnoldi eigenvalue solver for a reactor benchmark problem when energy decomposition is needed. The MG Krylov, MGE preconditioner, and RQI solver combination also scales well in energy. Finally, this solver set is a strong choice for very large and challenging problems.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

A Multilevel in Space and Energy Solver for 3-D Multigroup Diffusion and Coarse-Mesh Finite Difference Eigenvalue Problems

The Multilevel in Space and Energy Diffusion (MSED) method accelerates the iterative convergence of multigroup diffusion eigenvalue problems by performing work on lower-order equations with only one group and/or coarser spatial grids. It consists of two primary components: (1) a grey (one-group) diffusion eigenvalue problem that is solved via Wielandt-shifted power iteration (PI) and (2) a multigrid-in-space linear solver. In previous work, the efficiency of MSED was verified using Fourier analysis and numerical results from a one-dimensional multigroup diffusion code. Since that work, MSED has been implemented as a solver for the coarse-mesh finite difference (CMFD) system in the three-dimensional Michigan Parallel Characteristics Transport (MPACT) code. In this paper, the results from the implementation of MSED in MPACT are introduced, and the changes needed to make MSED more suitable for MPACT are described. For problems without feedback, the conclusions in this paper show that MSED can reduce the CMFD run time by an order of magnitude and the overall run time by a factor of 2 to 3 compared to the default CMFD solver in MPACT [PI with the generalized minimal residual (GMRES) method]. For problems with feedback, the convergence of the outer Picard iteration scheme is worsened by the well-converged CMFD solutions produced by the standard MSED method. To overcome this unintuitive deficiency, MSED may be run with looser convergence criteria (a modified version of the MSED method called MSED-L) to circumvent the issue until the multiphysics iteration in MPACT is improved. Results show that MSED-L can reduce the CMFD run time in MPACT by an order of magnitude, without negatively impacting the outer Picard iteration scheme.

3-D↗