Results for “scalable solvers” — Page 5

NASA NTRS2014

Using SpF to Achieve Petascale for Legacy Pseudospectral Applications

Pseudospectral (PS) methods possess a number of characteristics (e.g., efficiency, accuracy, natural boundary conditions) that are extremely desirable for dynamo models. Unfortunately, dynamo models based upon PS methods face a number of daunting challenges, which include exposing additional parallelism, leveraging hardware accelerators, exploiting hybrid parallelism, and improving the scalability of global memory transposes. Although these issues are a concern for most models, solutions for PS methods tend to require far more pervasive changes to underlying data and control structures. Further, improvements in performance in one model are difficult to transfer to other models, resulting in significant duplication of effort across the research community. We have developed an extensible software framework for pseudospectral methods called SpF that is intended to enable extreme scalability and optimal performance. Highlevel abstractions provided by SpF unburden applications of the responsibility of managing domain decomposition and load balance while reducing the changes in code required to adapt to new computing architectures. The key design concept in SpF is that each phase of the numerical calculation is partitioned into disjoint numerical kernels that can be performed entirely inprocessor. The granularity of domain decomposition provided by SpF is only constrained by the datalocality requirements of these kernels. SpF builds on top of optimized vendor libraries for common numerical operations such as transforms, matrix solvers, etc., but can also be configured to use open source alternatives for portability. SpF includes several alternative schemes for global data redistribution and is expected to serve as an ideal testbed for further research into optimal approaches for different network architectures. In this presentation, we will describe our experience in porting legacy pseudospectral models, MoSST and DYNAMO, to use SpF as well as present preliminary performance results provided by the improved scalability.

DYNAMO↗

DOE OSTI2024

Accelerating Multivariate Functional Approximation Computation with Domain Decomposition Techniques⋆

Modeling large datasets through Multivariate Functional Approximations (MFA) provide an elegant way to handle many visualization and scientific analysis workflows. The process necessitates scalable data partitioning methods to compute MFA representations efficiently without compromising the accuracy or continuity of the reconstructed solution. We propose a domain -decomposed method for computing the MFA with B -spline bases, which reduces the total work per task and uses a restricted Additive Schwarz (RAS) method to converge the control point data degrees -of -freedom along subdomain boundaries. We provide an in-depth analysis of the parallel approach with domain decomposition solvers, aiming to minimize local subdomain error residuals and recover high -order continuity at subdomain interfaces with appropriate choices of knot overlaps. The communication cost, determined by the overlap regions in the RAS implementation, is optimized to recover the numerical error profile of the single subdomain case. Our proposed method stands in contrast to previous methods, which typically only recover either C 0 or at best C 1 continuity for arbitrary B -spline degree expansions, or those that require post -processing to blend discontinuities in the reconstructed data. We demonstrate the effectiveness of our approach using analytical and real -world datasets in 1D, 2D, and 3D through both strong and weak scaling studies. The performance results indicate that the overall cost of computing the approximation is directly proportional to the underlying nearest -neighbor communication implementation, and is only weakly dependent on the overlap region size that determines the size of the messages. This finding underscores the efficiency and scalability of our proposed method, making it a promising solution for handling large datasets in scientific workflows.

additive Schwarz solvers↗

NASA NTRS2008

Scalability of Localized Arc Filament Plasma Actuators

Temporal flow control of a jet has been widely studied in the past to enhance jet mixing or reduce jet noise. Most of this research, however, has been done using small diameter low Reynolds number jets that often have little resemblance to the much larger jets common in real world applications because the flow actuators available lacked either the power or bandwidth to sufficiently impact these larger higher energy jets. The Localized Arc Filament Plasma Actuators (LAFPA), developed at the Ohio State University (OSU), have demonstrated the ability to impact a small high speed jet in experiments conducted at OSU and the power to perturb a larger high Reynolds number jet in experiments conducted at the NASA Glenn Research Center. However, the response measured in the large-scale experiments was significantly reduced for the same number of actuators compared to the jet response found in the small-scale experiments. A computational study has been initiated to simulate the LAFPA system with additional actuators on a large-scale jet to determine the number of actuators required to achieve the same desired response for a given jet diameter. Central to this computational study is a model for the LAFPA that both accurately represents the physics of the actuator and can be implemented into a computational fluid dynamics solver. One possible model, based on pressure waves created by the rapid localized heating that occurs at the actuator, is investigated using simplified axisymmetric simulations. The results of these simulations will be used to determine the validity of the model before more realistic and time consuming three-dimensional simulations are conducted to ultimately determine the scalability of the LAFPA system.

Brown, Clifford A.↗

NASA NTRS1997

A Comparison of PETSC Library and HPF Implementations of an Archetypal PDE Computation

Two paradigms for distributed-memory parallel computation that free the application programmer from the details of message passing are compared for an archetypal structured scientific computation a nonlinear, structured-grid partial differential equation boundary value problem using the same algorithm on the same hardware. Both paradigms, parallel libraries represented by Argonne's PETSC, and parallel languages represented by the Portland Group's HPF, are found to be easy to use for this problem class, and both are reasonably effective in exploiting concurrency after a short learning curve. The level of involvement required by the application programmer under either paradigm includes specification of the data partitioning (corresponding to a geometrically simple decomposition of the domain of the PDE). Programming in SPAM style for the PETSC library requires writing the routines that discretize the PDE and its Jacobian, managing subdomain-to-processor mappings (affine global- to-local index mappings), and interfacing to library solver routines. Programming for HPF requires a complete sequential implementation of the same algorithm, introducing concurrency through subdomain blocking (an effort similar to the index mapping), and modest experimentation with rewriting loops to elucidate to the compiler the latent concurrency. Correctness and scalability are cross-validated on up to 32 nodes of an IBM SP2.

Hayder, M. Ehtesham↗

NASA NTRS2024

Forced Response Aeromechanics Analysis in MATLAB®-Based Environment Code With Application to Distortion-Tolerant Fan R24 Blade Geometry

Researchers at the NASA Glenn Research Center have developed a research-level code for forced response analysis for turbomachinery that provides a streamlined framework for aeromechanics analysis, as well as for generating Goodman diagrams. The Forced Response Aeromechanics Analysis in a MATLAB-Based Environment (FRAAME, Version 1) code is designed to accept blade surface unsteady pressure time histories generated by three-dimensional (3D) Unsteady Reynolds-Averaged Navier-Stokes (URANS) code TURBO and modal displacements, modal stresses, and static stresses generated via expanded ANSYS® (Ansys, Inc.) cyclic symmetry analysis. The code’s looping structure allows for largescale analyses, including many blade surface pressure files, modal displacements, and modal stress files for full annulus forced response analysis, including a modal summation method for multiple engine orders and modes. This code is applied to the R24 fan geometry for the Boundary Layer Ingesting Inlet/Distortion-Tolerant Fan (BLI2DTF) project, a propulsion system being developed to increase the fuel efficiency of future aircraft, to serve as validation of experimental data as well as external simulation results. This tool’s functional, modular form is intended to allow users to make modifications easily. Those modifications can include requests for information for any step in the analysis process, as well as adding various functions to compute additional information of interest. It is also written, given sufficient input information, as a general blade geometry forced response solver that is not necessarily specific to the R24 geometry but is used here for verification of code functionality and accuracy. Although forced response analysis tools are also available in commercially available code packages, the post-processing FRAAME code offers reliable, fast, and scalable forced response analysis and Goodman diagram generation for large cases utilizing, specifically, TURBO and ANSYS® results information in an effort to streamline forced response and high cycle fatigue analyses.

Aeromechanics↗

DOE OSTI2024

A Finite Difference informed Random Walk solver for simulating radiation defect evolution in polycrystalline structures with strongly inhomogeneous diffusivity

Diffusivity of species and defects on grain boundaries is usually several orders of magnitude larger than that inside grains. Such strongly inhomogeneous diffusivity requires prohibitively high computational demands for modeling microstructural evolution. Here, this paper presents a highly-efficient numerical solver, combining the Finite Difference method and Random Walk model, designed for accurately modeling strongly inhomogeneous diffusion within polycrystalline structures. The proposed solver, termed Finite Difference informed Random Walk (FDiRW), integrates a customized Finite Difference (cFD) scheme tailored for fast diffusion along thin grain boundaries represented by a single-layer of nodes. Numerical experiments demonstrate that the FDiRW solver achieves an impressive efficiency gain of 1560x compared to traditional Finite Difference methods while maintaining accuracy, making it feasible for personal computer machines to handle diffusional systems with strongly inhomogeneous diffusivity across static polycrystalline microstructures. The model has been successfully applied to simulate radiation defect evolution, showcasing its scalability to engineering scales in both length and time dimensions.

36 MATERIALS SCIENCE↗

DOE OSTI2024

Matrix-Free High-Performance Saddle-Point Solvers for High-Order Problems in $\boldsymbol{H}(\operatorname{\textbf{div}})$

Here, this work describes the development of matrix-free GPU-accelerated solvers for high-order finite element problems in H(div). The solvers are applicable to grad-div and Darcy problems in saddle-point formulation, and have applications in radiation diffusion and porous media flow problems, among others. Using the interpolation–histopolation basis, efficient matrix-free preconditioners can be constructed for the (1, 1)-block and Schur complement of the block system. With these approximations, block-preconditioned MINRES converges in a number of iterations that is independent of the mesh size and polynomial degree. The approximate Schur complement takes the form of an M-matrix graph Laplacian and therefore can be well-preconditioned by highly scalable algebraic multigrid methods. High-performance GPU-accelerated algorithms for all components of the solution algorithm are developed, discussed, and benchmarked. Numerical results are presented on a number of challenging test cases, including the “crooked pipe” grad-div problem, the SPE10 reservoir modeling benchmark problem, and a nonlinear radiation diffusion test case.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

LuGo: An enhanced quantum phase estimation implementation

Quantum Phase Estimation (QPE) is a cardinal algorithm in quantum computing that plays a crucial role in various applications, including cryptography, molecular simulation, and solving systems of linear equations. However, the standard implementation of QPE faces challenges related to time complexity and circuit depth, which limit its practicality for large-scale computations. We introduce LuGo, a novel framework designed to enhance the performance of QPE by reducing circuit duplication, as well as using parallelization techniques to achieve faster generation of the QPE circuit and gate reduction. We validate the effectiveness of our framework by generating quantum linear solver circuits, which require both QPE and inverse QPE, to solve linear systems of equations. LuGo achieves significant improvements in both computational efficiency and hardware requirements without compromising on accuracy. Compared to a standard QPE implementation, LuGo reduces time consumption to generate a circuit that solves a 2 6 × 2 6 system matrix by a factor of 50.68 and over 31× reduction of quantum gates and circuit depth, with no fidelity loss on an ideal quantum simulator. Furthermore, we demonstrated the versatility and scalability of LuGo enabled HHL algorithm by simulating a canonical Hele-Shaw fluid problem using a quantum simulator. With these advantages, LuGo paves the way for more efficient implementations of QPE, enabling broader applications across several quantum computing domains.

Quantum algorithm↗

DOE OSTI2024

Constrained Local Approximate Ideal Restriction for Advection-Diffusion Problems

Herein this paper focuses on developing a reduction-based algebraic multigrid (AMG) method that is suitable for solving general (non)symmetric linear systems and is naturally robust from pure advection to pure diffusion. Initial motivation comes from a new reduction-based AMG approach, $\ell \text{AIR}$ (local approximate ideal restriction), that was developed for solving advection-dominated problems. Though this new solver is very effective in the advection-dominated regime, its performance degrades in cases where diffusion becomes dominant. This is consistent with the fact that in general, reduction-based AMG methods tend to suffer from growth in complexity and/or convergence rates as the problem size is increased, especially for diffusion-dominated problems in two or three dimensions. Motivated by the success of $\ell \text{AIR}$ in the advective regime, our aim in this paper is to generalize the AIR framework with the goal of improving the performance of the solver in diffusion-dominated regimes. To do so, we propose a novel way to combine mode constraints as used commonly in energy-minimization AMG methods with the local approximation of ideal operators used in $\ell \text{AIR}$. The resulting constrained $\ell \text{AIR}$ algorithm is able to achieve fast scalable convergence on advective and diffusive problems. In addition, it is able to achieve standard low complexity hierarchies in the diffusive regime through aggressive coarsening, something that was previously difficult for reduction-based methods.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2002

Progress in the Simulation of Steady and Time-Dependent Flows with 3D Parallel Unstructured Cartesian Methods

The proposed paper will present recent extensions in the development of an efficient Euler solver for adaptively-refined Cartesian meshes with embedded boundaries. The paper will focus on extensions of the basic method to include solution adaptation, time-dependent flow simulation, and arbitrary rigid domain motion. The parallel multilevel method makes use of on-the-fly parallel domain decomposition to achieve extremely good scalability on large numbers of processors, and is coupled with an automatic coarse mesh generation algorithm for efficient processing by a multigrid smoother. Numerical results are presented demonstrating parallel speed-ups of up to 435 on 512 processors. Solution-based adaptation may be keyed off truncation error estimates using tau-extrapolation or a variety of feature detection based refinement parameters. The multigrid method is extended to for time-dependent flows through the use of a dual-time approach. The extension to rigid domain motion uses an Arbitrary Lagrangian-Eulerlarian (ALE) formulation, and results will be presented for a variety of two- and three-dimensional example problems with both simple and complex geometry.

Aftosmis, M. J.↗

NASA NTRSDate not supplied

Validation of Loci-Stream for Autogenous Pressurization of Cryogenic Propellant Tank

Autogenous pressurization of cryogenic propellant tanks eliminates the need to have an additional pressurant tank on the space vehicle, which is highly advantageous due to reduced vehicle mass and design complexity. Autogenous pressurization therefore is one of the key technologies for deep space exploration and long-term space missions. The complex interaction of thermal gradients, turbulence and phase change near the interface make the problem a challenging one to model. Nodal analysis tools and reduced order models are unable to capture the necessary physics. 3-D CFD analyses are necessary to fully characterize autogenous pressurization. CFD analyses pose their own difficulties. The requisite CFD tool to tackle this problem need to be modular with the ability to incorporate various physics models, efficient, and computationally scalable for simulating flight size tanks. NASA MSFC's Loci-Stream CFD tool along with the VOF module is a great candidate to fit this mold. We demonstrate our modeling approach and validation of Loci-Stream for predicting autogenous pressurization of a flight scale propellant tank in order for the solver to serve as a reliable design and analysis tool for NASA's CFM application needs. Liquid hydrogen tank pressurization tests carried out at the MSFC test stand 300 facilities provide reliable validation data for this purpose. These tests were modeled using the Loci-Stream solver with a newly implemented two-phase sharp interface treatment. We show that our modeling approach and CFD solver predict the autogenous pressurization phenomena satisfactorily, and document challenging aspects of modeling this problem.

cryogenic fluid management↗

NASA NTRSDate not supplied

Validation of Loci-Stream for Autogenous Pressurization of Cryogenic Propellant Tank

Autogenous pressurization of cryogenic propellant tanks eliminates the need to have an additional pressurant tank on the space vehicle, which is highly advantageous due to reduced vehicle mass and design complexity. Autogenous pressurization therefore is one of the key technologies for deep space exploration and long-term space missions. The complex interaction of thermal gradients, turbulence and phase change near the interface make the problem a challenging one to model. Nodal analysis tools and reduced order models are unable to capture the necessary physics. 3-D CFD analyses are necessary to fully characterize autogenous pressurization. CFD analyses pose their own difficulties. The requisite CFD tool to tackle this problem need to be modular with the ability to incorporate various physics models, efficient, and computationally scalable for simulating flight size tanks. NASA MSFC's Loci-Stream CFD tool along with the VOF module is a great candidate to fit this mold. We demonstrate our modeling approach and validation of Loci-Stream for predicting autogenous pressurization of a flight scale propellant tank in order for the solver to serve as a reliable design and analysis tool for NASA's CFM application needs. Liquid hydrogen tank pressurization tests carried out at the MSFC test stand 300 facilities provide reliable validation data for this purpose. These tests were modeled using the Loci-Stream solver with a newly implemented two-phase sharp interface treatment. We show that our modeling approach and CFD solver predict the autogenous pressurization phenomena satisfactorily, and document challenging aspects of modeling this problem.

CFM↗

NASA NTRSDate not supplied

Validation of Two Phase Flow Modeling Techniques in Loci-Stream using Axial Jet Pressure Control and Autogenous Pressurization Experiment

Autogenous pressurization of cryogenic propellant tanks eliminates the need to have an additional pressurant tank on the space vehicle, which is highly advantageous due to reduced vehicle mass and design complexity. Autogenous pressurization therefore is one of the key technologies for deep space exploration and long-term space missions. The complex interaction of thermal gradients, turbulence and phase change near the interface make the problem a challenging one to model. Nodal analysis tools and reduced order models are unable to capture the necessary physics. 3-D CFD analyses are necessary to fully characterize autogenous pressurization. CFD analyses pose their own difficulties. The requisite CFD tool to tackle this problem need to be modular with the ability to incorporate various physics models, efficient, and computationally scalable for simulating flight size tanks. NASA MSFC's Loci-Stream CFD tool along with the VOF module is a great candidate to fit this mold. We demonstrate our modeling approach and validation of Loci-Stream for predicting autogenous pressurization of a flight scale propellant tank in order for the solver to serve as a reliable design and analysis tool for NASA's CFM application needs. Liquid hydrogen tank pressurization tests carried out at the MSFC test stand 300 facilities provide reliable validation data for this purpose. These tests were modeled using the Loci-Stream solver with a newly implemented two-phase sharp interface treatment. We show that our modeling approach and CFD solver predict the autogenous pressurization phenomena satisfactorily, and document challenging aspects of modeling this problem.

cryogenic fluid management↗

DOE OSTI2026

Integrated Transmission-Distribution Multi-Period Switching for Wildfire Risk Mitigation: Improving Speed and Scalability with Distributed Optimization: Preprint

With increasingly severe wildfire conditions driven by climate change, utilities must manage the risk of wildfire ignitions from electric power lines. During "public safety power shutoff'" events, utilities de-energize power lines to reduce wildfire ignition risk, which may result in load shedding. Distributed energy resources provide flexibility that can help support the system to reduce load shedding when lines are de-energized. We investigate a coordinated transmission-distribution optimization problem that balances wildfire risk mitigation and load shedding. We model distribution systems that include battery energy storage systems which may support loads when transmission lines are de-energized. This multi-period integrated transmission-distribution optimal switching problem jointly optimizes line switching decisions, the generators' setpoints, load shedding, and the batteries' states of charge, resulting in significant computational challenges. To improve scalability, we decompose the problem over both space and time and apply a distributed optimization algorithm. Using a large-scale synthetic California test case with realistic distribution models and real wildfire risk data, we show that distributed optimization can solve large-scale multi-period switching problems that are otherwise intractable for centralized solvers. We also discuss challenges and future directions for improving the distributed algorithm's convergence performance as the number of time periods increases.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS2004

AIC Computations Using Navier-Stokes Equations on Single Image Supercomputers For Design Optimization

A procedure to accurately generate AIC using the Navier-Stokes solver including grid deformation is presented. Preliminary results show good comparisons between experiment and computed flutter boundaries for a rectangular wing. A full wing body configuration of an orbital space plane is selected for demonstration on a large number of processors. In the final paper the AIC of full wing body configuration will be computed. The scalability of the procedure on supercomputer will be demonstrated.

Guruswamy, Guru↗

DOE OSTI2024

Towards exascale for wind energy simulations

We examine large-eddy-simulation modeling approaches and computational performance of two open-source computational fluid dynamics codes for the simulation of atmospheric boundary layer flows that are of direct relevance to wind energy production. The first code, NekRS, is a high-order, unstructured-grid, spectral element code. The second code, AMR-Wind, is a second-order, block-structured, finite-volume code with adaptive mesh refinement capabilities. The objective of this study is to co-develop these codes in order to improve model fidelity and performance for each. These features will be critical for running ABL-based applications such as wind farm analysis on advanced computing architectures. To this end, we investigate the performance of NekRS and AMR-Wind on the Oak Ridge Leadership Facility supercomputers Summit, using 4 to 800 nodes (24 to 4,800 NVIDIA V100 GPUs), and Crusher, the testbed for the Frontier exascale system, using 18 to 384 Graphics Compute Dies on AMD MI250X GPUs. We compare strong- and weak-scaling capabilities, linear solver performance, and time to solution. We also identify leading inhibitors to parallel scaling.

17 WIND ENERGY↗

DOE OSTI2026

Development of MOSCATO: A CFD-Level Electrochemistry and Corrosion Simulator for Molten Salt Systems

For both coolant and fueled variants of molten salt reactors (MSRs), the corrosion of structural materials is a significant challenge. The corrosion stems from chemical and electrochemical reactions initiated by fissile material, fission products, and impurities in the salt. Lower-fidelity models rely on empirical correlations for mass transfer, simplified lumped temperature profiles, and similar assumptions. They do not capture detailed spatial variations in complex geometries, creating the need for high-fidelity modeling to bridge this gap.As we approach the demonstration and possible deployment of MSRs in this decade, the development of a high-fidelity, high-performance simulator becomes imperative. To simulate the complex electrochemical environment and corrosion within molten salt systems, we have developed the Molten Salt Chemistry And TranspOrt (MOSCATO) code. This endeavor is comprised of three essential components. First, mass transfer equations are coupled with the Navier-Stokes equations in order to account for the transport of species in the salt. Second, the diffusion of alloy constituents, such as Cr, Fe, Ni, etc. is simulated within the structural metals. Third, the alloy and salt domains are coupled to account for the heterogeneous chemical and electrochemical reactions that occur at the salt-alloy interface.MOSCATO manages all three components within the framework of the highly scalable, open-source spectral element method computational fluid dynamics code Nek5000/NekRS. This integration enables MOSCATO to harness the immense computational power of modern high-performance computing resources, ensuring both high fidelity and computational speed.In addition to code development, we have initiated a comprehensive verification and validation campaign, utilizing data from diverse sources. First, MOSCATO's electrochemical solver was verified with reference numerical data. Then validation occurred against experiments: one of a thermal galvanic cell and the other for corrosion in flowing molten salt of FLiNaK (LiF-NaF-KF). This campaign verified and validated MOSCATO as a reliable tool for simulating electrochemical environments and corrosion in molten salt systems.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2025

Randomized Preconditioned Solvers for Strong Constraint 4D-Var Data Assimilation

The Strong Constraint 4D Variational (SC-4DVAR) data assimilation method is widely used in climate and weather applications. SC-4DVAR involves solving a minimization problem to compute the maximum a posteriori estimate, which we tackle using the Gauss-Newton method. The computation of the descent direction is expensive since it involves the solution of a large-scale and potentially ill-conditioned linear system, solved using the preconditioned conjugate gradient (PCG) method. Here, to address this cost, we efficiently construct scalable preconditioners using three different randomization techniques, which all rely on a certain low-rank structure involving the Gauss-Newton Hessian. The proposed techniques come with theoretical guarantees on the condition number, and at the same time, are amenable to parallelization. We also develop an adaptive approach to estimate the sketch size and choose between the reuse or recomputation of the preconditioner. We demonstrate the performance and effectiveness of our methodology on two representative model problems—the Burgers and barotropic vorticity equation—showing a drastic reduction in both the number of PCG iterations and the number of Gauss-Newton Hessian products after including the preconditioner construction cost.

Gauss-Newton↗