Results for “parallel codes” — Page 10

DOE OSTI2022

Sierra/SolidMechanics 5.8 User's Manual

Sierra/SolidMechanics (Sierra/SM) is a Lagrangian, three-dimensional code for finite element analysis of solids and structures. It provides capabilities for explicit dynamic, implicit quasistatic and dynamic analyses. The explicit dynamics capabilities allow for the efficient and robust solution of models with extensive contact subjected to large, suddenly applied loads. For implicit problems, Sierra/SM uses a multi-level iterative solver, which enables it to effectively solve problems with large deformations, nonlinear material behavior, and contact. Sierra/SM has a versatile library of continuum and structural elements, and a large library of material models. The code is written for parallel computing environments enabling scalable solutions of extremely large problems for both implicit and explicit analyses. It is built on the SIERRA Framework, which facilitates coupling with other SIERRA mechanics codes. This document describes the functionality and input syntax for Sierra/SM.

42 ENGINEERING↗

DOE OSTI2022

Sierra/SolidMechanics 5.10 User's Guide

Sierra/SolidMechanics (Sierra/SM) is a Lagrangian, three-dimensional code for finite element analysis of solids and structures. It provides capabilities for explicit dynamic, implicit quasistatic and dynamic analyses. The explicit dynamics capabilities allow for the efficient and robust solution of models with extensive contact subjected to large, suddenly applied loads. For implicit problems, Sierra/SM uses a multi-level iterative solver, which enables it to effectively solve problems with large deformations, nonlinear material behavior, and contact. Sierra/SM has a versatile library of continuum and structural elements, and a large library of material models. The code is written for parallel computing environments enabling scalable solutions of extremely large problems for both implicit and explicit analyses. It is built on the SIERRA Framework, which facilitates coupling with other SIERRA mechanics codes. This document describes the functionality and input syntax for Sierra/SM.

42 ENGINEERING↗

DOE OSTI2023

Sierra/Solid Mechanics 5.16 User's Guide

Sierra/SolidMechanics (Sierra/SM) is a Lagrangian, three-dimensional code for finite element analysis of solids and structures. It provides capabilities for explicit dynamic, implicit quasistatic and dynamic analyses. The explicit dynamics capabilities allow for the efficient and robust solution of models with extensive contact subjected to large, suddenly applied loads. For implicit problems, Sierra/SM uses a multi-level iterative solver, which enables it to effectively solve problems with large deformations, nonlinear material behavior, and contact. Sierra/SM has a versatile library of continuum and structural elements, and a large library of material models. The code is written for parallel computing environments enabling scalable solutions of extremely large problems for both implicit and explicit analyses. It is built on the SIERRA Framework, which facilitates coupling with other SIERRA mechanics codes. This document describes the functionality and input syntax for Sierra/SM.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Evaluating Portable Parallelization Strategies for Heterogeneous Architectures in High Energy Physics

High-energy physics (HEP) experiments have developed millions of lines of code over decades that are optimized to run on traditional x86 CPU systems. However, we are seeing a rapidly increasing fraction of floating point computing power in leadership-class computing facilities and traditional data centers coming from new accelerator architectures, such as GPUs. HEP experiments are now faced with the untenable prospect of rewriting millions of lines of x86 CPU code, for the increasingly dominant architectures found in these computational accelerators. This task is made more challenging by the architecture-specific languages and APIs promoted by manufacturers such as NVIDIA, Intel and AMD. Producing multiple, architecture-specific implementations is not a viable scenario, given the available person power and code maintenance issues. The Portable Parallelization Strategies team of the HEP Center for Computational Excellence is investigating the use of Kokkos, SYCL, OpenMP, std::execution::parallel and alpaka as potential portability solutions that promise to execute on multiple architectures from the same source code, using representative use cases from major HEP experiments, including the DUNE experiment of the Long Baseline Neutrino Facility, and the ATLAS and CMS experiments of the Large Hadron Collider. This cross-cutting evaluation of portability solutions using real applications will help inform and guide the HEP community when choosing their software and hardware suites for the next generation of experimental frameworks. We present the outcomes of our studies, including performance metrics, porting challenges, API evaluations, and build system integration.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2024

CI/CD Efforts for Validation, Verification and Benchmarking OpenMP Implementations

Software developers must adapt to keep up with the changing capabilities of platforms so that they can utilize the power of High-Performance Computers (HPC), including exascale systems. OpenMP, a directive-based parallel programming model, allows developers to include directives to existing C, C++, or Fortran code to allow node level parallelism without compromising performance. This paper describes our CI/CD efforts to provide easy evaluation of the support of OpenMP across different compilers using existing testsuites and benchmark suites on HPC platforms. Our main contributions include (1) the set of a Continuous Integration (CI) and Continuous Development (CD) workflow that captures bugs and provides faster feedback to compiler developers, (2) an evaluation of OpenMP (offloading) implementations supported by AMD, HPE, GNU, LLVM, and Intel, and (3) evaluation of the quality of compilers across different heterogeneous HPC platforms. With the comprehensive testing through the CI/CD workflow, we aim to provide a comprehensive understanding of the current state of OpenMP (offloading) support in different compilers and heterogeneous platforms consisting of CPUs and GPUs from NVIDIA, AMD, and Intel.

Jarmusch, Aaron↗

DOE OSTI2022

The elphbolt ab initio solver for the coupled electron-phonon Boltzmann transport equations

elphbolt is a modern Fortran (2018 standard) code for efficiently solving the coupled electron–phonon Boltzmann transport equations from first principles. Using results from density functional and density functional perturbation theory as inputs, it can calculate the effect of the non-equilibrium phonons on the electronic transport (phonon drag) and non-equilibrium electrons on the phononic transport (electron drag) in a fully self-consistent manner and obeying the constraints mandated by thermodynamics. It can calculate the lattice, charge, and thermoelectric transport coefficients for the temperature gradient and electric fields, and the effect of the mutual electron–phonon drag on these transport properties. The code fully exploits the symmetries of the crystal and the transport-active window to allow the sampling of extremely fine electron and phonon wave vector meshes required for accurately capturing the drag phenomena. The corray feature of modern Fortran, which offers native and convenient support for parallelization, is utilized. The code is compact, readable, well-documented, and extensible by design.

electron drag↗

DOE OSTI2021

Hydrogen Component Leak Rate Quantification for System Risk and Reliability Assessment through QRA and PHM Frameworks: Preprint

The National Renewable Energy Laboratory's (NREL) Hydrogen Safety Research and Development (HSR&D) program in collaboration with the University of Maryland's Systems Risk and Reliability Analysis Laboratory (SyRRA) are working to improve reliability and reduce risk in hydrogen systems. This approach strives to use quantitative data on component leaks and failures, together with Prognosis and Health Management (PHM), and Quantitative Risk Assessment (QRA) to identify at-risk components, reduce component failures and downtime, and predict when components require maintenance. Hydrogen component failures increase facility maintenance cost, facility downtime, and reduce public acceptance of hydrogen technologies, ultimately increasing facility size and cost because of potentially overly conservative requirements. Leaks are a predominant failure mode for hydrogen components. However, uncertainties in the amount of hydrogen emitted from leaking components and the frequency of those failure events limit the understanding of the risks that they present under real-world operational conditions. NREL has deployed a test fixture, the Leak Rate Quantification Apparatus (LRQA), to quantify the mass flow rate of leaking gases from medium and high-pressure components that have failed while in service. Quantitative hydrogen leak rate data from this system could ultimately be used to better inform risk assessment and Regulation Codes and Standards (RCS). Parallel activity explores the use of PHM and QRA techniques to assess and reduce risk, thereby improving safety and reliability of hydrogen systems. The results of QRAs could further provide a systematic and science-based foundation for the design and implementation of RCS, as in the latest versions of the NFPA 2 code for gaseous hydrogen stations. Alternatively, data-driven techniques of PHM could provide new damage diagnosis and health-state prognosis tools. This research will help end users, station owners and operators, and regulatory bodies move towards risk-informed preventative maintenance versus emergency corrective maintenance, reducing cost and improving reliability. Predictive modelling of failures could improve safety and affect RCS requirements such as setback distances at liquid refuelling sites. The combination of leak rate quantification research, PHM, and QRA can lead to better informed models enabling data-based decision to be made for hydrogen system safety improvements.

codes and standards↗

DOE OSTI2018

Center for Extended Magnetohydrodynamic Modeling

The RPI team efforts focused on providing unstructured mesh technologies to the Center for Extended Magnetohydrodynamic Modeling’s efforts on the development of the M3D-C1 plasma physics code. Specific developments included: Providing the parallel unstructured mesh infrastructure used within the M3D-C1 code; Creating a tool for the generation and subsequent adaptation of the unstructured meshes; A set of procedures to be able to initiate an M3D-C1 simulation in 2D and automatically moving the simulation to a full 3D finite element model when 3D modes begin to appear; Developing a set of physics/mesh based diagonal preconditioners; Supporting the installation of M3D-C1 on various super computer systems; Supplementing the mesh generation procedures to support a model wall region to support the application of an advanced resistive wall model; Providing M3D-C1 with tools to support PIC operations on distributed meshes.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2019

Harnessing billions of tasks for a scalable portable hydrodynamic simulation of the merger of two stars

We propose a highly scalable demonstration of a portable asynchronous many-task programming model and runtime system applied to a grid-based adaptive mesh refinement hydrodynamic simulation of a double white dwarf merger with 14 levels of refinement that spans 17 orders of magnitude in astrophysical densities. The code uses the portable C++ parallel programming model that is embodied in the HPX library and being incorporated into the ISO C++ standard. The model reflects a significant shift from existing bulk synchronous parallel programming models under consideration for exascale systems. Through the use of the Futurization technique, seemingly sequential code is transformed into wait-free asynchronous tasks. We demonstrate the potential of our model by showing results from strong scaling runs on National Energy Research Scientific Computing Center’s Cori system (658,784 Intel Knight’s Landing cores) that achieve a parallel efficiency of 96.8% using billions of asynchronous tasks.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

RF-transpond: A 1D coupled cold plasma wave and plasma transport model for ponderomotive force driven density modification parallel to B

The RF-Transpond code couples a fluid plasma transport solver with a frequency domain cold plasma RF wave solver in a 1D domain parallel to a strong background magnetic field. A ponderomotive force term proportional to parallel gradients in the electric field strength is included in the transport model in order to describe ponderomotive effects in the scrape-off layer (SOL) of fusion plasmas. The transport and wave codes are verified independently and a coupled case corresponding to experimental parameters from the LArge Plasma Device (LAPD) is presented. The density perturbation ratio R n , calculated to describe ponderomotive force driven modifications, is up to 20% for the simulation inputs used.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2025

OpenSn: A massively parallel, open-source simulation environment for discrete ordinates radiation transport

OpenSn is an open-source, massively parallel deterministic radiation transport code for solving the discrete-ordinates ( S N ) form of the Boltzmann transport equation on unstructured, arbitrary polyhedral meshes. It supports high-fidelity simulations involving steady-state, eigenvalue, and adjoint problems for neutral particles (e.g., neutrons, photons, multi-particles), using the multigroup approximation in energy. OpenSn combines angular discretization via discrete ordinates with a discontinuous Galerkin finite element method (DGFEM) in space, enabling accurate resolution of transport physics on arbitrary polyhedral cells, included locally refined spatial grids. It includes multiple angular quadrature types, including locally refined angular quadratures. Written in modern C++ with a Python API, OpenSn runs efficiently on platforms ranging from laptops to supercomputers. The transport sweep algorithm is implemented using a task-based, directed-acyclic-graph (DAG) approach for each angle and supports asynchronous parallelism across thousands of MPI ranks. Group-set aggregation improves compute intensity, and synthetic acceleration techniques (e.g., diffusion synthetic acceleration, second-moment method) enhance solver convergence. OpenSn has been verified on reactor physics problems and demonstrated excellent weak and strong scaling performance on more than 32,768 processes, making it a versatile and robust platform for large-scale transport simulations in complex geometries.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2019

Direct Numerical Simulation of Hypersonic Turbulent Boundary Layer Flow using SPARC: Initial Evaluation

This report documents the initial testing of the Sandia Parallel Aerodynamics and Reentry Code (SPARC) to directly simulate hypersonic, turbulent boundary layer flow over a sharp 7- degree half-angle cone. This type of computation involves a tremendously large range of scales both in time and space, requiring a large number of grid cells and the efficient utilization of a large pool of resources. The goal of the simulation is to mimic and verify a wind tunnel experiment that seeks to measure the turbulent surface pressure fluctuations. These data are necessary for building a model to predict random vibration loading in the reentry flight environment. A low-dissipation flux scheme in SPARC is used on a 2.7 billion cell mesh to capture the turbulent fluctuations in the boundary layer flow. The grid is divided into 115200 partitions and simulated using the Knight's Landings (KNL) partition of the Trinity system. The parallel performance of SPARC is explored on the Trinity system, as well as some of the other new architectures. Extracting data from the simulation shows good agreement with the experiment as well as a colleague's simulation. The data provide a guide for which a new model can be built for better prediction of the reentry random vibration loads.

42 ENGINEERING↗

DOE OSTI2020

Impact of moderate pump–Stokes chirp on femtosecond coherent anti-Stokes Raman scattering spectra

The effects of inducing moderate chirp in the pump and Stokes beams in chirped-probe-pulse femtosecond coherent anti-Stokes Raman scattering (CARS) spectroscopy are discussed. This is an important issue for measurements in high-pressure systems where the CARS beams must be transmitted through glass of significant thickness to reach the probe volume of interest. The effects were investigated both experimentally, by inserting disks of SF-11 glass into the pump and Stokes beam paths, and theoretically, by incorporating pulse chirp into our time-dependent density matrix (TDDM) simulations. Experimentally, we found that inducing moderate pump and Stokes chirp produced significant spectral narrowing of the nonresonant four-wave mixing contribution to the CARS signal, and this allowed us to control the resonant excitation bandwidth to eliminate interferences from neighboring species. The temperature measurement accuracy and precision were essentially unchanged with respect to the unchirped pump and Stokes case. The effect of moderate pump and Stokes chirp on the narrowing of the Raman excitation efficiency envelope was investigated by solving the full set of TDDM through direct numerical integration. The numerical efficiency of the solutions was enhanced dramatically by developing a parallel version of the TDDM code. The calculated Raman excitation efficiency profile was incorporated in our phenomenological spectral fitting codes.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2020

Modelling of nanometer scale dust grains in tokamak

Dust poses a serious threat to tokamak operation and safety. It is important to study the behaviour of dust grains under tokamak's discharge conditions, which depends heavily on their size and charge. Existing simulations mainly address issues on dust grains with radii larger than 1 μm, in which case, the drift effect due to electromagnetic fields can be safely ignored. For nanometer scale dust grains, however, the drift effect becomes significant and a new model based on guiding-centre system needs to be established. In this work, the NDS has been done under BOUT++ framework. The simulation contains two parts. Part one, NDS evaluates the charging and ablation processes of the dust grains. In the second part, the guiding-centre orbits of dust particles are tracked in tokamak plasmas, whose parameters are obtained from BOUT++, a highly desirable C++ code package for performing parallel plasma fluid simulations with an arbitrary number of equations in 3D curvilinear coordinates. The orbit of nanodust dynamics is described by guiding centre equations for simplicity, and these equations are numerically solved by conventional fourth-order Runge Kutta method. Simulations provide results such as trajectories and evolutions of dust particles with different sizes and velocities for different tokamak geometries. Results show tungsten dust grains with a radius of a few nanometers launched from outer midplane will oscillate before totally ablated in C-Mod. The oscillation in this case is driven by the ion drag force. Larger Nanodust with a radius of 100 nm, on the contrary, cannot be completely constrained by the electromagnetic field. In conclusion, the high plasma temperature and density in the seperatrix region causes severe dust ablation, resulting in total ablation within several ms.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2018

OpenMP in VASP: Threading and SIMD

Abstract The Vienna Ab initio Simulation Package (VASP) is a widely used electronic structure code that originally exploits process‐level parallelism through the Message Passing Interface (MPI) for work distribution within and across nodes. Architectural changes of modern parallel processors urge programmers to address thread‐ and data‐level parallelism as well to benefit most from the available compute resources within a node. We describe for VASP how to approach for an MPI + OpenMP parallelization including data‐level parallelism through OpenMP SIMD constructs together with a generic high‐level vector coding scheme. We can demonstrate an improved scalability of VASP and more than 20% gain over the MPI‐only version as well as a 2× increased performance of collective operations using the multiple‐endpoint MPI feature. The high‐level vector coding scheme applied to VASP's general gradient approximation routine gives up 9× performance gain on AVX512 platforms with the Intel compiler.

Wende, Florian↗

DOE OSTI2020

Fluid Dynamics Effects on Microstructure Prediction in Single-Laser Tracks for Additive Manufacturing of IN625

Single-track laser fusion were simulated using a heat-transfer-solidification-only (HTS) model and its extension with fluid dynamics (HTS_FD) model using a parallel open-source code, which included laminar fluid dynamics, flat-free surface of the molten alloy, heat transfer, phase-change, evaporation, and surface tension phenomena. The results illustrate that the fluid dynamics affects the solidification and ensuing microstructure. For the HTS_FD simulations, thermal gradient, G was found to exhibit a maximum at the extremity of the solidified pool ( i.e. , at the free surface), while for HTS simulations, G exhibited a maximum around the entire edge of the solidified pool. HTS_FD simulations predicted a wider range of cooling rates than the HTS simulations, exhibited an increased spread in the solidification speed, V variation within the melt-pool with respect to the HTS model results. Primary dendrite arm spacing (PDAS) were evaluated based on power law correlations and marginal stability theory models using the ( G , V ) from HTS and HTS_FD simulations to quantify the effect of the fluid dynamics on the microstructure. At low-laser powers and low-scan speeds, the PDAS obtained with the fluid dynamics model (HTS_FD) was larger by more than 30 pct with respect to the PDAS calculated with the simple HTS model. A new PDAS correlation, i.e. , \( \lambda_{1} \left[ {\mu {\text{m}}} \right] = 832\;G\left[ {\text{K/m}} \right]^{ - 0.5} V\left[ {\text{m/s}} \right]^{ - 0.25} \) , which uses the ( G , V ) results from the HTS_FD model was developed and validated against experimental results.

36 MATERIALS SCIENCE↗

DOE OSTI2025

A nonhydrostatic formulation for MPAS-Ocean

The Model for Prediction Across Scales-Ocean (MPAS-Ocean) is an open-source, global ocean model and is one component of a family of climate models within the MPAS framework, including atmosphere, sea-ice, and land-ice models. Here, in this work, a new formulation for the ocean model is presented that solves the nonhydrostatic, incompressible Boussinesq equations on an unstructured, staggered, z-level grid. The introduction of this nonhydrostatic capability is necessary for the resolution of internal wave dynamics and large eddy simulations. Compared to the standard, hydrostatic formulation, a nonhydrostatic pressure solver and a vertical momentum equation are added, where the PETSc (Portable Extensible Toolkit for Scientific Computation) library is used for the inversion of a large sparse system for the nonhydrostatic pressure. Numerical results on a stratified seiche, internal solitary wave, overflow and lock-exchange test cases are presented, and the parallel efficiency of the code is evaluated using up to 1024 processors.

3D Poisson equation↗

DOE OSTI2025

A nonhydrostatic formulation for MPAS-Ocean

The Model for Prediction Across Scales-Ocean (MPAS-Ocean) is an open-source, global ocean model and is one component of a family of climate models within the MPAS framework, including atmosphere, sea-ice, and land-ice models. Here, in this work, a new formulation for the ocean model is presented that solves the nonhydrostatic, incompressible Boussinesq equations on an unstructured, staggered, z-level grid. The introduction of this nonhydrostatic capability is necessary for the resolution of internal wave dynamics and large eddy simulations. Compared to the standard, hydrostatic formulation, a nonhydrostatic pressure solver and a vertical momentum equation are added, where the PETSc (Portable Extensible Toolkit for Scientific Computation) library is used for the inversion of a large sparse system for the nonhydrostatic pressure. Numerical results on a stratified seiche, internal solitary wave, overflow and lock-exchange test cases are presented, and the parallel efficiency of the code is evaluated using up to 1024 processors.

3D Poisson equation↗