Search NASA⌕ Search

SEARCH · Search NASA

Results for “SCALE Code System”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

Extreme Load Computational Fluid Dynamics Analysis and Verification for a Multibody Wave Energy Converter

A wave energy converter (WEC) must be designed to survive the extreme sea states that it will be subject to throughout its lifetime. Although there are many analysis methods and codes available to accomplish this, there are currently several engineering challenges to WEC survival design. Foremost, the computational design approach will typically involve a trade-off between accuracy and computational efficiency. Additionally, most computational fluid dynamics (CFD) codes are not ideally suited to modeling extreme events for WECs with multibody dynamics, power-take-off systems, and mooring systems. Finally, although WEC design standards and CFD guidelines are emerging, with the current immaturity of the WEC industry, they are not yet well established. In this study, loads on a 1:35-scale, moored, multibody WEC are evaluated with CFD. The CFD results are compared with results obtained from a computationally efficient, midfidelity model based on linearized potential flow hydrodynamics. For these model verification comparisons, both operational and survival configurations are considered. The extreme load results obtained, using both codes, indicate that the survival configuration successfully sheds loads during extreme sea states. It is also found that WEC-Sim, when appropriately applied, can provide reasonable load results, at a fraction of the computational expense of CFD. However, for the more extreme sea states, and for higher-order effects not included in the WEC-Sim model, the linear-based results have significant errors in comparison to the CFD-based results, and should be used judiciously.

16 TIDAL AND WAVE POWER↗

Extreme Load Computational Fluid Dynamics Analysis and Verification for a Multibody Wave Energy Converter: Preprint

A wave energy converter (WEC) must be designed to survive the extreme sea states that it will be subject to throughout its lifetime. Although there are many analysis methods and codes available to accomplish this, there are currently several engineering challenges to WEC survival design. Foremost, the computational design approach will typically involve a trade-off between accuracy and computational efficiency. Additionally, most computational fluid dynamics (CFD) codes are not ideally suited to modeling extreme events for WECs with multibody dynamics, power-take-off systems, and mooring systems. Finally, although WEC design standards and CFD guidelines are emerging, with the current immaturity of the WEC industry, they are not yet well established. In this study, loads on a 1:35-scale, moored, multibody WEC are evaluated with CFD. The CFD results are compared with results obtained from a computationally efficient, midfidelity model based on linearized potential flow hydrodynamics. For these model verification comparisons, both operational and survival configurations are considered. The extreme load results obtained, using both codes, indicate that the survival configuration successfully sheds loads during extreme sea states. It is also found that WEC-Sim, when appropriately applied, can provide reasonable load results, at a fraction of the computational expense of CFD. However, for the more extreme sea states, and for higher-order effects not included in the WEC-Sim model, the linear-based results have significant errors in comparison to the CFD-based results, and should be used judiciously.

16 TIDAL AND WAVE POWER↗

A review of future weather data for assessing climate change impacts on buildings and energy systems

The effectiveness of climate change impact assessments and the development of adaptation strategies depend on the availability of high-quality future weather data. However, significant gaps exist between the needs of the energy research community and the focus of the climate modeling community, primarily due to a historical lack of communication and collaboration between the two groups. Here, to address this issue, this work provides a comprehensive overview of the critical aspects involved in creating future weather data for building and energy system modeling, including emissions scenarios, general circulation models, downscaling methods, categories of future weather data, and uncertainties in climate simulations. Moreover, it critically evaluates the applicability and suitability of various types of future weather data in five key application scenarios: energy use analysis, resilience analysis, HVAC design, utility-scale analysis, and renewable energy analysis. Finally, this work presents recommendations for high-level actions and research directions to foster collaboration between the energy research and climate modeling communities and to promote the integration of future weather data into energy codes and the design practices of buildings and energy systems.

Climate change↗

Quantitative comparisons of electron-scale turbulence measurements in NSTX via synthetic diagnostics for high-k scattering.

Two synthetic diagnostics are implemented for the high-k scattering system in NSTX [D. R. Smith, E. Mazzucato, W. Lee, H. K. Park, C. W. Domier, and N. C. Luhmann Jr., Rev. Sci. Instrum., 79, 123501 (2008)] allowing direct comparisons between synthetic and experimentally detected frequency and wavenumber spectra of electron-scale turbulence fluctuations. Synthetic diagnostics were formulated in real-space and in wavenumber space, and were deployed to realistic electron-scale simulations carried out with the GYRO code [J. Candy and R. E. Waltz, J. Comput. Phys. 186 (2003) 545]. A highly unstable ETG regime (electron temperature gradient mode) in a modest beta NSTX NBI-heated H-mode discharge was chosen for the analysis. Mapping the measured wavenumbers to field aligned coordinates showed that the high-k system is sensitive to fluctuations that are closer to the spectral peak in the density fluctuation wavenumber spectrum (streamers) than originally predicted. Analysis of synthetic spectra show that the frequency response of the detected fluctuations is dominated by Doppler shift and insensitive to the turbulence drive. The shape of the high-k density fluctuation wavenumber spectrum is sensitive to the ETG turbulence drive conditions, and can be reproduced in a sensitivity scan of the most pertinent turbulent drive terms in the simulation.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Integrated Optimization and Control of a Hybrid Gas Turbine/sCO 2 Power System

During phase-I, the project team led by Echogen Power Systems (EPS) had two primary objectives based on investigating the application of gas turbines with supercritical carbon dioxide (sCO 2 ) power cycles. The first objective was to improve the overall efficiency and performance of a hybrid gas turbine/sCO 2 power system through a joint optimization of the two subsystems (gas turbine and sCO 2 power cycle) using non-linear optimization techniques that simultaneously evaluate thermal performance of the combined cycle. The hybrid power system included several points of interaction, including (but not limited to) gas turbine exhaust, fuel heating, inlet chilling and turbine cooling. The second objective was to establish a baseline transient response model of the hybrid power system and a notional microgrid and begin steps to integrate the control systems of the three major elements (gas turbine, sCO 2 cycle and grid controller). The project team established a baseline performance for a combined cycle power plant using a production gas turbine and scaled sCO 2 power cycle only utilizing exhaust heat recovery. Echogen’s non-linear techno-economic optimization code was extended by adding gas turbine component models derived from a in-house developed gas turbine design code. With the two cycles coupled by the gas turbine exhaust, design parameters of both cycles were allowed to vary simultaneously to determine performance opportunity versus isolated designs. Returning to the baseline gas turbine/sCO 2 power cycle transient models: Echogen had in-house developed sCO 2 cycle transient model in GT-Suite system simulation software, and had partnered with Siemens Finspång for gas turbine transient model, and Siemens PTI group to provide micro-grid load profile as well as hybrid power cycle generated load (power and frequency) analysis. The transient model for the SGT-750 Siemens gas turbine was a “black-box” functional mock-up interface (FMI) model developed by Siemens Industrial Turbomachinery in Finspång, Sweden. The SGT-750 is a twin-shaft gas turbine that produces 40 MW electricity with an efficiency of about 40% at ISO conditions. At 100% gas turbine throttle (load), the SGT-750 has average exhaust conditions of 114.6 kg/s and 469.8°C. The transient model for sCO 2 power cycle was developed by Echogen in GT-SUITE 1D system simulation software platform. The basic CO 2 flow circuit has single-shaft turbomachinery with net 11.5 MW electrical power output at design conditions. The power turbine has a double-ended shaft with one end connected to synchronous generator through a fixed-ratio gearbox. The other end of power turbine is connected to the compressor through a continuously variable transmission. The major components of the sCO 2 power cycle modeled include air cooled condenser/cooler, CO 2 compressor, recuperator, two waste heat exchanger coils, power turbine, continuous variable transmission, gearbox and generator. Integration of SGT-750 transient model and sCO 2 power cycle transient model was done in Matlab Simulink. In the integrated model, the gas turbine and sCO 2 power cycle interacted at two points, first one being the gas turbine exhaust gas flow rate and temperature, which were inputs to sCO 2 power cycle model. The second point was the distribution of micro-grid load demand signal between the SGT-750 generator and sCO 2 cycle generator. For a given combined-cycle load demand, the gas turbine load demand was equal to the total demand minus the sCO 2 cycle power generated. In the present study the integrated model was simulated for two cases of grid load demand: (i) for a step change, both positive-step and negative-step, in grid load demand (ii) for a micro-grid load demand curve provided by Siemens PTI group. Finally, the time series plots representing load demand versus integrated system response were presented including the sCO 2 power cycle control system performance plots. The actual generated power and frequency of both the generators, gas turbine and sCO 2 power cycle, was supplied to Siemens PTI group for dynamic grid assessment, results of which are provided in appendices.

03 NATURAL GAS↗

Toward Performance Portable Programming for Heterogeneous System-on-Chips: Case Study with Qualcomm Snapdragon SoC

Future heterogeneous Domain-Specific System-on-Chips (DSSoC) will be extraordinarily complex in terms of processors, memory hierarchies, and interconnection networks.To manage this complexity, architects, system software designers, and application developers need programming technologies that are flexible, accurate, efficient, and productive. These technologies will need to be as independent of any one specific architecture as is practical, because the sheer dimensionality and scale of the complexity will not allow porting and optimizing applications foreach given DSSoC. To address these issues, we are developing Cosmic Castle, a performance portable programming toolchain for streaming applications on heterogeneous architectures. The primary focus of Cosmic Castle is on enabling efficient and performant code generation through the smart compiler and intelligent runtime system. This paper presents the preliminary evaluation of our ongoing work toward Cosmic Castle. Specifically, we detail our code porting efforts and evaluate various benchmarks on the Qualcomm Snapdragon SoC using tools developed through Cosmic Castle.

Cabrera, Anthony↗

Toward a standardization of cryostructure and cryogenic soil structure terminology for the field description of permafrost‐affected soils

This paper establishes standardized terminology and field documentation protocols for cryostructures and cryogenic soil structures in permafrost‐affected soils and provides brief guidance on descriptions of ground ice morphology and ice volume estimates. We consolidate permafrost terminology from Russian and North American literature, clarify long‐standing ambiguities, and provide explicit guidelines that align with US Department of Agriculture‐Natural Resources Conservation Service soil description standards. Our scheme makes critical distinctions between cryostructure, the distribution of ice within soil, and cryogenic soil structure, the morphological structure of soil resulting from ice formation. The scheme organizes cryostructures into three main categories: non‐segregated ice, visible segregated ice, and ice matrices. We introduce standardized codes and parameters for field descriptions of ice and soil that enable machine‐readable data collection compatible with existing soil information systems. This standardization will significantly enhance the integration of field observations into landscape‐scale assessments of permafrost stability, infrastructure vulnerability, and ecosystem response to permafrost thaw, addressing an urgent need for quantitative data to inform modeling and decision‐making in rapidly changing Arctic and subarctic environments.

Andersen, Megan L. [University of Minnesota, Saint↗

MFC 5.0: An exascale many-physics flow solver

Many problems of interest in engineering, medicine, and the fundamental sciences rely on high-fidelity flow simulation, making performant computational fluid dynamics solvers a mainstay of the open-source software community. Previous work MFC 3.0 was made a published, documented, and open-source solver via Bryngelson et al. Comp. Phys. Comm. (2021) with numerous physical features, numerical methods, and scalable infrastructure. MFC 5.0 is a significant update to MFC 3.0, featuring a broad set of well-established and novel physical models and numerical methods, as well as the introduction of GPU and APU (or superchip) acceleration. Here, we exhibit state-of-the-art performance and ideal scaling on the first two exascale supercomputers, OLCF Frontier and LLNL El Capitan. Combined with MFC’s single-accelerator performance, MFC achieves exascale computation in practice, and achieved the largest-to-date public CFD simulation at 200 trillion grid points as a 2025 ACM Gordon Bell Prize finalist. New physical features include the immersed boundary method, N-fluid phase change, Euler–Euler and Euler–Lagrange sub-grid bubble models, fluid-structure interaction, hypo- and hyper-elastic materials, chemically reacting flow, two-material surface tension, magnetohydrodynamics (MHD), and more. Numerical techniques now represent the current state-of-the-art, including general relaxation characteristic boundary conditions, WENO variants, Strang splitting for stiff sub-grid flow features, and low Mach number treatments. Weak scaling to tens of thousands of GPUs on OLCF Summit and Frontier and LLNL El Capitan achieves efficiencies within 5% of ideal to over 90% of their respective system sizes. Strong scaling results for a 16-times increase in device count show parallel efficiencies over 90% on OLCF Frontier. MFC’s software stack has undergone further improvements, including continuous integration, which ensures code resilience and correctness through over 300 regression tests; metaprogramming, which reduces code length while maintaining performance portability; and code generation for computing chemical reactions

Computational fluid dynamics↗

Openpronghorn

OpenPronghorn is a simulation tool specifically tailored for modeling thermal-hydraulic phenomena in advanced nuclear reactors. It is built on the Multiphysics Object-Oriented Simulation Environment (MOOSE), an open-source platform that facilitates the development of high-performance scientific computing applications. OpenPronghorn solves the Navier-Stokes equations, which describe the conservation of mass, momentum, and energy in fluid flows, using the finite volume numerical method. The code supports a wide range of fluid flow conditions that are applicable to nuclear reactors, including incompressible and weakly compressible flows, as well as single-phase and multiphase flows. It is capable of modeling diverse flow regimes, including laminar and turbulent flows, using various turbulence models such as the standard k-epsilon models, the v2f model, and the mixing length model. For multiphase flows, OpenPronghorn employs a mixture a Eulerian modeling approach with mixture, drift-flux, and full Eulerian models, and includes open-sourced interfacial transfer correlations for drag, exchange, and heat transfer coming from the scientific literature. OpenPronghorn's modular design allows it to handle multiscale simulations, ranging from detailed Reynolds-Averaged Navier Stokes (RANS) simulations to coarse-mesh and lumped parameter models. This flexibility enables users to perform high-fidelity simulations of specific reactor components as well as system-level analyses of entire reactor circuits. The code can be coupled with other MOOSE-based tools using the MultiApp system, allowing for the transfer of coupling quantities such as mass flow rates, heat fluxes, and boundary conditions between different simulation scales. One of the main features of OpenPronghorn is the it includes built-in validation cases from the open-source scientific literature and supports the implementation of user-defined models and correlations through MOOSE's FunctorMaterial system. OpenPronghorn is designed to be computationally efficient, leveraging the SIMPLE projection method for large-scale problems, and can be run on high-performance computing systems to handle the extensive computational demands of detailed reactor simulations. Overall, OpenPronghorn is a versatile and robust tool that provides critical insights into the thermal-hydraulic behavior of advanced nuclear reactors, supporting the design, safety, and optimization of next-generation nuclear energy systems.

Retamales, Mauricio Eduardo Tano [Idaho National L↗

STEPS: A Portable Numerical Simulation Toolkit for Electrical Power System Dynamic Studies

Numerical simulation is the key technique for large scale power system analysis. Redistribution of global renewable power via international interconnections requires new simulation tools to study the interconnected systems with different nominal frequencies as a whole. In this paper we introduce an open source simulation toolkit for electrical power systems (STEPS) which is hosted at Github. Its kernel is coded in C++ with major functions of power flow and electro-mechanical dynamic simulation. Flexible options are provided and configurable to improve power flow solution and dynamic simulation. Common devices and models are supported in STEPS for AC/DC hybrid system studies. Studies of interconnected systems with different nominal frequencies is supported in STEPS for research of international interconnection. Application program interfaces are provided and wrapped with Python to enable high-level interfaces for general applications. STEPS is thread safe and parallel computation is supported in both kernel and script levels to accelerate simulation. It is portable and works on Windows and GNU/Linux platforms. Cases from small to large scale systems are thoroughly tested to validate the toolkit with commercial packages as benchmarks.

42 ENGINEERING↗

How to Build a Quantum Supercomputer: Scaling from Hundreds to Millions of Qubits

In the span of four decades, quantum computation has evolved from an intellectual curiosity to a potentially realizable technology. Today, small-scale demonstrations have become possible for quantum algorithmic primitives on hundreds of physical qubits and proof-of-principle error-correction on a single logical qubit. Nevertheless, despite significant progress and excitement, the path toward a full-stack scalable technology is largely unknown. There are significant outstanding quantum hardware, fabrication, software architecture, and algorithmic challenges that are either unresolved or overlooked. These issues could seriously undermine the arrival of utility-scale quantum computers for the foreseeable future. Here, we provide a comprehensive review of these scaling challenges. We show how the road to scaling could be paved by adopting existing semiconductor technology to build much higher-quality qubits, employing system engineering approaches, and performing distributed quantum computation within heterogeneous high-performance computing infrastructures. These opportunities for research and development could unlock certain promising applications, in particular, efficient quantum simulation/learning of quantum data generated by natural or engineered quantum systems. To estimate the true cost of such promises, we provide a detailed resource and sensitivity analysis for classically hard quantum chemistry calculations on surface-code error-corrected quantum computers given current, target, and desired hardware specifications based on superconducting qubits, accounting for a realistic distribution of errors. Furthermore, we argue that, to tackle industry-scale classical optimization and machine learning problems in a cost-effective manner, heterogeneous quantum-probabilistic computing with custom-designed accelerators should be considered as a complementary path toward scalability.

Mohseni, Masoud↗

Validation of Numerical Models of the Offshore Wind Turbine From the Alpha Ventus Wind Farm Against Full-Scale Measurements Within OC5 Phase III

The main objective of the Offshore Code Comparison Collaboration Continuation, with Correlation (OC5) project is validation of aero-hydro-servo-elastic simulation tools for offshore wind turbines (OWTs) through comparison of simulated results to the response data of physical systems. Phase III of the OC5 project validates OWT models against the measurements recorded on a Senvion 5M wind turbine supported by the OWEC Quattropod from the alpha ventus offshore wind farm. The following operating conditions of the wind turbine were chosen for the validation: (1) Idling below the cut-in wind speed; (2) Rotor-nacelle assembly (RNA) rotation maneuver below the cut-in wind speed; (3) Power production below and above the rated wind speed; and (4) Shutdown. A number of validation load cases were defined based on these operating conditions. The following measurements were used for validation: (1) Strains and accelerations recorded on the support structure; (2) Pitch, yaw, and azimuth angles, generator speed, and electrical power recorded from the RNA. Strains were not directly available from the majority of the OWT simulation tools. Therefore, strains were calculated based on out-of-plane bending moments, axial forces, and cross-sectional properties of the structural members. Also, a number of issues arose during the validation: (1) The need for a thorough quality check of sensor measurements; (2) The sensitivity of the turbine loads to the controller and airfoil properties, which were only approximated in the modeling approach; (3) The importance of estimating and applying an appropriate damping value for the structure; and (4) The importance of wind characteristics beyond turbulence on the loads. The simulation results and measurements were compared in terms of time series, discrete Fourier transforms, power spectral densities, probability density functions of strains and accelerometers. A good match was achieved between the measurements and models set up by OC5 Phase III participants.

17 WIND ENERGY↗

IRIS: Exploring Performance Scaling of the Intelligent Runtime System and its Dynamic Scheduling Policies

High-Performance Computing is becoming increasingly heterogeneous, relying on a diverse mix of hardware to achieve good performance. Paradoxically, current drivers and frameworks for these devices typically require separate languages and implementations for each vendor. Furthermore, there are few tools and little support to schedule codes between these devices in a truly heterogeneous manner-partly because of this fragmentation between vendors and the languages each supports. To overcome both limitations, the Intelligent Runtime System (IRIS) was developed. It allows a common task abstraction to automatically be shared among contemporary vendors and is run from a single host-side API. At runtime, IRIS queries the host system and registers which frameworks and drivers are available, these determine which kernels can be used by the scheduler-CPUs via OpenMP, Nvidia GPUs (CUDA), AMD GPUs (HIP), and Intel and Xilinx FPGAs with OpenCL. IRIS enables tasks to be scheduled to any heterogeneous device and resolves to the appropriate kernel binary at runtimeit only uses the devices supported by the system on which it is run. IRIS supports single-task and graph-based expressions of dependencies of tasks. Additionally, IRIS features a range of dynamic scheduling policies, allowing complex chains of tasks and interactions to be executed, relieving the programmer/user from considering the system to assign tasks to devices optimally. This paper presents the peak performance attainable by IRIS over a range of systems-each with different numbers and types of accelerator devices, it highlights the flexibility of IRIS since these devices are truly heterogeneous, relying on different backends (drivers, frameworks, and languages) which historically required unique implementations to utilize them. We then use this peak performance as a baseline to compare increasingly complex chains of tasks (with increasingly complex task dependencies) and evaluate how IRIS copes. Finally, we consider the performance of different IRIS scheduling policies on this range of task graphs.

Johnston, Beau↗

Experimental characterization and modeling of optical tweezer particle handling dynamics [Modeling and Simulation of Optical Tweezer Particle Handling Dynamics]

Here, we report a new framework for a quantitative understanding of optical trapping (OT) particle handling dynamics. We present a novel three-dimensional particle-based model that includes optical, hydrodynamic, and inter-particle forces. This semi-empirical colloid model is based on an open-source simulation code known as LAMMPS (large-scale atomic/molecular massively parallel simulator) and properly recapitulates the full OT force profile beyond the typical linear approximations valid near the trap center. Simulations are carried out with typical system parameters relevant for our experimental holographic optical trapping (HOT) system, including varied particle sizes, trap movement speeds, and beam powers. Furthermore, we present a new experimental method for measuring both the stable and metastable boundaries of the optical force profile to inform or validate the model’s underlying force profile. We show that our framework is a powerful tool for accurately predicting particle behavior in a practical experimental OT setup and can be used to characterize and predict particle handling dynamics within any arbitrary OT force profile.

36 MATERIALS SCIENCE↗

The Outer Rim Simulation: A Path to Many-core Supercomputers

Herein we describe the Outer Rim cosmological simulation, one of the largest high-resolution N-body simulations performed to date, aimed at promoting science to be carried out with large-scale structure surveys. The simulation covers a volume of (4.225 Gpc) 3 and evolves more than one trillion particles. It was executed on Mira, a BlueGene/Q system at the Argonne Leadership Computing Facility. We discuss some of the computational challenges posed by a system such as Mira, a many-core supercomputer, and how the simulation code, Hardware/Hybrid Accelerated Cosmology Code, has been designed to overcome these challenges. We have carried out a large range of analyses on the simulation data and we report on the results as well as the data products that have been generated. The full data set generated by the simulation totals more than 5 PB of data, making curation and handling of the data a large challenge in itself. The simulation results have been used to generate synthetic catalogs for large-scale structure surveys, including Dark Energy Spectroscopic Instrument and eBOSS, as well as experiments on the cosmic microwave background. A detailed catalog for the data challenges of the Large Synoptic Survey Telescope Dark Energy Science Collaboration has been created as well. We publicly release some of the Outer Rim halo catalogs, downsampled particle information, and lightcone data.

79 ASTRONOMY AND ASTROPHYSICS↗

Validation of Numerical Models of the Offshore Wind Turbine from the Alpha Ventus Wind Farm Against Full-Scale Measurements Within OC5 Phase III: Preprint

The main objective of the Offshore Code Comparison Collaboration Continuation, with Correlation (OC5) project, is validation of aero-hydro-servo-elastic simulation tools for offshore wind turbines (OWTs) through comparison of simulated results to the response data of physical systems. Phase III of the OC5 project validates OWT models against the measurements recorded on Senvion 5M wind turbine supported by the OWEC Quattropod from the alpha ventus offshore wind farm. The following operating conditions of the wind turbine were chosen for the validation: (1) Idling below the cut-in wind speed; (2) Rotornacelle assembly (RNA) rotation maneuver below the cut-in wind speed; (3) Power production below and above the rated wind speed; and (4) Shut-down. A number of validation load cases were defined based on these operating conditions. The following measurements were used for validation: (1) Strains and accelerations recorded on the support structure; (2) Pitch, yaw, and azimuth angles, generator speed, and electrical power recorded from the RNA. Strains were not directly available from the majority of the OWT simulation tools. Therefore, strains were calculated based on out-of-plane bending moments, axial forces, and geometrical properties of the structural members. Also, a number of issues arose during the validation: (1) The need for a thorough quality check of sensor measurements; (2) The sensitivity of the turbine loads to the controller and airfoil properties, which were only approximated in the modeling approach; (3) The importance of estimating and applying an appropriate damping value for the structure; and (4) The importance of wind characteristics beyond turbulence on the loads. The simulation results and measurements were compared in terms of time series, discrete Fourier transforms (DFTs), power spectral densities (PSDs), probability density functions (PDFs) of strains and accelerometers. A good match was achieved between the measurements and models set up by OC5 Phase III participants.

17 WIND ENERGY↗

Compare linear-system solver and preconditioner stacks with emphasis on GPU performance and propose phase-2 NGP solver development pathway

The goal of the ExaWind project is to enable predictive simulations of wind farms comprised of many megawatt-scale turbines situated in complex terrain. Predictive simulations will require computational fluid dynamics (CFD) simulations for which the mesh resolves the geometry of the turbines and captures the rotation and large deflections of blades. Whereas such simulations for a single turbine are arguably petascale class, multi-turbine wind farm simulations will require exascale-class resources. The primary physics codes in the ExaWind project are Nalu-Wind, which is an unstructured-grid solver for the acoustically incompressible Navier-Stokes equations, and OpenFAST, which is a whole-turbine simulation code. The Nalu-Wind model consists of the mass-continuity Poisson-type equation for pressure and a momentum equation for the velocity. For such modeling approaches, simulation times are dominated by linear-system setup and solution for the continuity and momentum systems. For the ExaWind challenge problem, the moving meshes greatly affect overall solver costs as reinitialization of matrices and recomputation of preconditioners is required at every time step. In this report we evaluated GPU-performance baselines for the linear solvers in the Trilinos and hypre solver stacks using two representative Nalu-Wind simulations: an atmospheric boundary layer precursor simulation on a structured mesh, and a fixed-wing simulation using unstructured overset meshes. Both strong-scaling and weak-scaling experiments were conducted on the OLCF supercomputer Summit and similar proxy clusters. We focused on the performance of multi-threaded Gauss-Seidel and two-stage Gauss-Seidel that are extensions of classical Gauss-Seidel; of one-reduce GMRES, a communication-reducing variant of the Krylov GMRES; and algebraic multigrid methods that incorporate the afore-mentioned methods. The team has established that AMG methods are capable of solving linear systems arising from the fixed-wing overset meshes on CPU, a critical intermediate result for ExaWind FY20 Q3 and Q4 milestones. For the fixed-wing strong-scaling study (model with 3M grid-points), the team identified that Nalu-Wind simulations with the new Trilinos and hypre solvers scale to modest GPU counts, maintaining above 70% efficiency up to 6 GPUs. However, there still remain significant bottlenecks to performance: matrix assembly (hypre), AMG setup (hypre and Trilinos) In the weak-scaling experiments (going from 0.4M to 211M gridpoints), it's shown that the solver apply phases are faster on GPUs, but that Nalu-Wind simulation times grow, primarily due to the multigrid-setup process. Finally, based on the report outcomes, we propose a linear solver path-forward for the remainder of the ExaWind project. Near term, the NREL team will continue their work on GPU-based linear-system assembly. They will also investigate how the use of alternatives to the NVIDIA UVM (unified virtual memory) paradigm affects performance. Longer term, the NREL team will evaluate algorithmic performance on other types of accelerators and merge their improvements back to the main hypre repository branch. Near term, the Trilinos team will address performance bottlenecks identified in this milestone, such as implementing a GPU-based segregated momentum solve and reusing matrix graphs across linear-system assembly phases. Longer term, the Trilinos team will do detailed analysis and optimization of multigrid setup.

17 WIND ENERGY↗