Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed parallelization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

Optimizing the Weather Research and Forecasting Model with OpenMP Offload and Codee

Currently, the Weather Research and Forecasting model (WRF) utilizes shared memory (OpenMP) and distributed memory (MPI) parallelisms. To take advantage of GPU resources on the Perlmutter supercomputer at NERSC, we port parts of the computationally expensive routine Fast Spectral Bin Microphysics (FSBM) to NVIDIA GPUs using OpenMP device offloading directives. To facilitate this process, we explore a workflow for optimization which uses both runtime profilers and a static code inspection tool Codee to refactor the subroutine. We observe an 2.24x overall speedup for the CONUS-12km storm test case.

Wichitrnithed, Chayanon (Namo) [Odin Institute]↗

Massively parallel direct numerical simulations of forced compressible turbulence: a hybrid MPI/OpenMP approach. In:XSEDE '12 Proceedings of the 1st Conference of the Extreme Science and Engineering Discovery Environment: Bridging from the eXtreme to the campus and beyond, Article No. 23

A highly scalable simulation code for turbulent flows which solves the fully compressible Navier-Stokes equations is presented. The code, which supports one, two and three dimensional domain decompositions is shown to scale well on up to 262,144 cores. Introducing multiple levels of parallelism based on distributed message passing and shared-memory paradigms results in a reduction of up to 33% of communication time at large core counts. The code has been used to generate a large database of homogeneous isotropic turbulence in a stationary state created by forcing the largest scales in the flow. The scaling of spectra of velocity and density fluctuations are presented. While the former follow classical theories strictly valid for incompressible flows, the latter presents a more complicated behavior. Fluctuations in velocity gradients and derived quantities exhibit extreme though rare fluctuations, a phenomenon known as intermittency. The simulations presented provide data to disentangle Reynolds and Mach number effects.

Jagannathan, Shriram↗

Optimizing the Four-Index Integral Transform Using Data Movement Lower Bounds Analysis

The four-index integral transform is a fundamental and computationally demanding calculation used in many computational chemistry suites such as NWChem. It transforms a four-dimensional tensor from an atomic basis to a molecular basis. This transformation is most efficiently implemented as a sequence of four tensor contractions that each contract a four-dimensional tensor with a two-dimensional transformation matrix. Differing degrees of permutation symmetry in the intermediate and final tensors in the sequence of contractions cause intermediate tensors to be much larger than the final tensor and limit the number of electronic states in the modeled systems. Loop fusion, in conjunction with tiling, can be very effective in reducing the total space requirement, as well as data movement. However, the large number of possible choices for loop fusion and tiling, and data/computation distribution across a parallel system, make it challenging to develop an optimized parallel implementation for the four-index integral transform. We develop a novel approach to address this problem, using lower bounds modeling of data movement complexity. We establish relationships between available aggregate physical memory in a parallel computer system and ineffective fusion configurations, enabling their pruning and consequent identification of effective choices and a characterization of optimality criteria. This work has resulted in the development of a significantly improved implementation of the four-index transform that enables higher performance and the ability to model larger electronic systems than the current implementation in the NWChem quantum chemistry software suite.

Rajbhandari, Samyam↗

OpenSHMEM over MPI-3

OpenSHMEM is a community effort to unifyt and standardize the SHMEM programming model. MPI (Message Passing Interface) is a well-known community standard for parallel programming using distributed memory. The most recen t release of MPI, version 3.0, was designed in part to support programming models like SHMEM.OSHMPI is an implementation of the OpenSHMEM standard using MPI-3 for the Linux operating system. It is the first implementation of SHMEM over MPI one-sided communication and has the potential to be widely adopted due to the portability and widely availability of Linux and MPI-3. OSHMPI has been tested on a variety of systems and implementations of MPI-3, includingInfiniBand clusters using MVAPICH2 and SGI shared-memory supercomputers using MPICH. Current support is limited to Linux but may be extended to Apple OSX if there is sufficient interest. The code is opensource via https://github.com/jeffhammond/oshmpi

Sayan Ghosh, JeffHammond↗

SSCA1-K1

SSCA1-K1 is a parallel implementation of kernel 1 of the SSCA1 benchmark suite released by the DARPA HPCS program. This kernel is able to run in parallel on a distributed shared memory system at extreme scales using OpenSHMEM.

Baker, Matthew↗

Quandary

Quandary numerically simulates and optimizes the time-evolution of open quantum systems. The underlying dynamics are modelled by Lindblad's master equation, a linear ordinary differential equation (ODE) describing quantum systems interacting with the environment. Quandary solves this ODE numerically by applying a time-stepping integration scheme, and utilizes a gradient-based optimization approach to determine optimal control pulses that drive the quantum system to a desired target state. Two optimization objectives are considered: (a) Unitary gate optimization that finds controls to realize a unitary gate transformation, and (b) optimal reset that aims to drive the quantum system to the ground states. Gradient-based optimization schemes utilizing Petsc's Tao optimization package are applied to generate control pulses that minimize the respective measure. To evaluate the gradient of the objective function, the discrete adjoint method is used while leveraging techniques from Algorithmic Differentiation to produce exact and consistent gradients. To mitigate excessive execution run times, the software can be build together with the XBraid software library which provides a parallelization strategy to distribute the time-evolution of the underlying dynamics onto multiple processor.

Petersson, NilsA.↗

Multipoint sensing with a low-coherence source using single-arm frequency-shifted interferometry

We demonstrate that multiple-site sensing along an optical fiber can be done with incoherent continuous-wave light. Here, using a broadband low-coherence noise source, a slow detector, and an optical modulator, we construct a single-arm frequency-shifted interferometer (SA-FSI) capable of simultaneously sensing multiple weak-reflection sites distributed either in parallel or in series along fiber links. By scanning the driving frequency of an electro-optic amplitude modulator in the range of 2.7–3.2 GHz at steps of 41.7 KHz, we demonstrate a spatial resolution of 0.3 m and a measurement range of over 1 km.

36 MATERIALS SCIENCE↗

ZettaBricks: A Language Compiler and Runtime System for Anyscale Computing

This grant supported the ZettaBricks and OpenTuner projects. ZettaBricks is a new implicitly parallel language and compiler where defining multiple implementations of multiple algorithms to solve a problem is the natural way of programming. ZettaBricks makes algorithmic choice a first class construct of the language. Choices are provided in a way that also allows our compiler to tune at a finer granularity. The ZettaBricks compiler autotunes programs by making both fine-grained as well as algorithmic choices. Choices also include different automatic parallelization techniques, data distributions, algorithmic parameters, transformations, and blocking. Additionally, ZettaBricks introduces novel techniques to autotune algorithms for different convergence criteria. When choosing between various direct and iterative methods, the ZettaBricks compiler is able to tune a program in such a way that delivers near-optimal efficiency for any desired level of accuracy. The compiler has the flexibility of utilizing different convergence criteria for the various components within a single algorithm, providing the user with accuracy choice alongside algorithmic choice. OpenTuner is a generalization of the experience gained in building an autotuner for ZettaBricks. OpenTuner is a new open source framework for building domain-specific multi-objective program autotuners. OpenTuner supports fully-customizable configuration representations, an extensible technique representation to allow for domain-specific techniques, and an easy to use interface for communicating with the program to be autotuned. A key capability inside OpenTuner is the use of ensembles of disparate search techniques simultaneously; techniques that perform well will dynamically be allocated a larger proportion of tests.

97 MATHEMATICS AND COMPUTING↗

FleCSPH notes

FleCSI is a compile-time con gurable framework designed to support multi-physics application development. As such, FleCSI provides a very general set of infrastructure design patterns that can be specialized and extended to suit the needs of a broad variety of solver and data requirements. FleCSI currently supports multi-dimensional mesh topology, geometry, and adjacency information, as well as n-dimensional hashed-tree data structures, graph partitioning interfaces, and dependency closures. FleCSI introduces a functional programming model with control, execution, and data abstractions that are consistent both with MPI and with state-of-the-art, task-based runtimes such as Legion and Charm++. The abstraction layer insulates developers from the underlying runtime, while allowing support for multiple runtime systems including conventional models like asynchronous MPI. The intent is to provide developers with a concrete set of user-friendly programming tools that can be used now, while allowing exibility in choosing runtime implementations and optimization that can be applied to future architectures and runtimes. FleCSI's control and execution models provide formal nomenclature for describing poorly understood concepts such as kernels and tasks. FleCSI's data model provides a low-buy-in approach that makes it an attractive option for many application projects, as developers are not locked into particular layouts or data structure representations. FleCSI currently provides a parallel but not distributed implementation of Binary, Quad and Oct-tree topology. This implementation is base on space lling curves domain decomposition, the Morton order. The current FleCSI version requires the implementation of a driver and a specialization driver. The role of the specialization driver is to provide the data distribution. This feature is not complete in FleCSI code and we provide it. The next step will be to incorporate it directly from FleCSPH to FleCSI as we reach a good level of performance. Then the driver represent the general execution of the resolution without worrying of the data locality and communications. As FleCSI is an On-Development code the structure may change in the future and we keep track of these changes in FleCSPH.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Final Technical Report: Application of in situ Neutron Diffraction to Understand the Mechanism of Phase Transitions

In this research, phase transitions in the bulk electrodes for Li-ion batteries were investigated using neutron diffraction (ND) as well as neutron imaging techniques. The objectives of this research is to design of a novel in situ electrochemical cell to obtain Rietveld refinable neutron diffraction experiments using small volume electrodes of various laboratory/research-scale electrodes intended for Li-ion batteries. This cell is also to be used to investigate the complexity of phase transitions in Li(Mg) alloy electrodes, either by diffraction or by neutron imaging, which occur under electrochemical lithiation and delithiation, and to determine aspects of phase transition that enable/limit energy storage capacity. Additional objective is to investigate the phase transitions in electrodes made of etched micro-columns of silicon and investigate the effect of particle/column size on phase transitions and nonequilibrium structures. An in situ electrochemical cell was designed successfully and was used to study the phase transitions under in-situ neutron diffraction in both the electrodes (anode/cathode) simultaneously in graphite/LiCoO 2 and in graphite/LiMn 2 O 4 cells each with two cells. The diffraction patterns fully validated the working of the in situ cell. Additional experimental were performed using the Si micro-columnar electrodes. The results revealed new lithiation phenomena, as evidenced by mosaicity formation in silicon electrode. These experiments were performed in Vulcan diffractometer at SNS, Oak Ridge National Laboratory. In parallel, the spatial distribution of Li during lithiation and delithiation processes in Li-battery electrodes were investigated. For this purpose, neutron tomographic imaging technique has been used for 3D mapping of Li distribution in bulk Li(Mg) alloy electrodes. It was possible to observe the phase boundary of Li(Mg) alloy indicating phase transition from Li-rich BCC β-phase to Li-lean α-phase. These experiments have been performed at CG-1D Neutron Imaging Prototype Station at SNS.

25 ENERGY STORAGE↗

Red Storm Case Study

Sandia and Cray Inc. co-developed Red Storm, a distributed memory, massively paralleled highperformance supercomputer modeled on ASCI Redl, to run computer codes used for conducting materials science simulations for national security. Supercomputers have some of the fastest highperformance systems available and are used primarily for scientific and engineering work requiring exceedingly high-speed computations. Unlike conventional computers, supercomputers have large storage capacity; more than one central processing unit to rapidly retrieve stored data and program instructions; and input/output capability.

97 MATHEMATICS AND COMPUTING↗

Decomposition and Algorithmic Approaches for Solving Large-Scale Process Family Design Problems

Our most recent work expands the water desalination case study from 76 variants to 10,897 variants using the equation-oriented model built in Pyomo as part of the PARETO project. Using the discretization formulation presented in Stinchfield (2024a), rather than solving for all 10,897 variants simultaneously, we decompose the formulation into subproblems containing subsets of variants from the process family. We solve the overall problem with Progressive Hedging (PH) deployed in parallel on a distributed HPC cluster using the open-source Python package mpi-sppy (Knueven et al., 2023). This approach allowed us to solve this process family design problem to ~1.5% relative optimality gap in about 5 hours; in comparison, Gurobi reached ~50% relative optimality gap in about 6 hours (Stinchfield et al., 2024b). However, this approach still requires discretization of the common unit module design ranges; additionally, PH acts as a heuristic for MILP’s with gap-closing capabilities. Ideally, we would not have to use ML surrogates or discretization to solve this problem, instead solving the process family design problem with the equation-oriented model directly to achieve the most accurate results. However, recall that we did not consider solving the MINLP directly due to complexity and size. In this work, we aim to decompose and solve this large-scale MINLP using a Structured Nonlinear Global Optimization algorithm presented by Cao and Zavala (2019).

Stinchfield, Georgia↗

Portals 4 network API definition and performance measurement

Portals is a low-level network programming interface for distributed memory massively parallel computing systems designed by Sandia, UNM, and Intel. Portals has been designed to provide high message rates and to provide the flexibility to support a variety of higher-level communication paradigms. This project developed and analyzed an implementation of Portals using shared memory in order to measure and understand the impact of using general-purpose compute cores to handle network protocol processing functions. The goal of this study was to evaluate an approach to high-performance networking software design and hardware support that would enable important DOE modeling and simulation applications to perform well and to provide valuable input to Intel so they can make informed decisions about future network software and hardware products that impact DOE applications.

99 GENERAL AND MISCELLANEOUS↗

Optimal control system design of an acid gas removal unit for an IGCC power plants with CO2 capture

Future IGCC plants with CO{sub 2} capture should be operated optimally in the face of disturbances without violating operational and environmental constraints. To achieve this goal, a systematic approach is taken in this work to design the control system of a selective, dual-stage Selexol-based acid gas removal (AGR) unit for a commercial-scale integrated gasification combined cycle (IGCC) power plant with pre-combustion CO{sub 2} capture. The control system design is performed in two stages with the objective of minimizing the auxiliary power while satisfying operational and environmental constraints in the presence of measured and unmeasured disturbances. In the first stage of the control system design, a top-down analysis is used to analyze degrees of freedom, define an operational objective, identify important disturbances and operational/environmental constraints, and select the control variables. With the degrees of freedom, the process is optimized with relation to the operational objective at nominal operation as well as under the disturbances identified. Operational and environmental constraints active at all operations are chosen as control variables. From the results of the optimization studies, self-optimizing control variables are identified for further examination. Several methods are explored in this work for the selection of these self-optimizing control variables. Modifications made to the existing methods will be discussed in this presentation. Due to the very large number of candidate sets available for control variables and due to the complexity of the underlying optimization problem, solution of this problem is computationally expensive. For reducing the computation time, parallel computing is performed using the Distributed Computing Server (DCS®) and the Parallel Computing® toolbox from Mathworks®. The second stage is a bottom-up design of the control layers used for the operation of the process. First, the regulatory control layer is designed followed by the supervisory control layer. Finally, an optimization layer is designed. In this paper, the proposed two-stage control system design approach is applied to the AGR unit for an IGCC power plant with CO{sub 2} capture. Aspen Plus Dynamics® is used to develop the dynamic AGR process model while MATLAB is used to perform the control system design and for implementation of model predictive control (MPC).

97 MATHEMATICS AND COMPUTING↗

Sensor placement algorithm development to maximize the efficiency of acid gas removal unit for integrated gasifiction combined sycle (IGCC) power plant with CO2 capture

Future integrated gasification combined cycle (IGCC) power plants with CO{sub 2} capture will face stricter operational and environmental constraints. Accurate values of relevant states/outputs/disturbances are needed to satisfy these constraints and to maximize the operational efficiency. Unfortunately, a number of these process variables cannot be measured while a number of them can be measured, but have low precision, reliability, or signal-to-noise ratio. In this work, a sensor placement (SP) algorithm is developed for optimal selection of sensor location, number, and type that can maximize the plant efficiency and result in a desired precision of the relevant measured/unmeasured states. In this work, an SP algorithm is developed for an selective, dual-stage Selexol-based acid gas removal (AGR) unit for an IGCC plant with pre-combustion CO{sub 2} capture. A comprehensive nonlinear dynamic model of the AGR unit is developed in Aspen Plus Dynamics® (APD) and used to generate a linear state-space model that is used in the SP algorithm. The SP algorithm is developed with the assumption that an optimal Kalman filter will be implemented in the plant for state and disturbance estimation. The algorithm is developed assuming steady-state Kalman filtering and steady-state operation of the plant. The control system is considered to operate based on the estimated states and thereby, captures the effects of the SP algorithm on the overall plant efficiency. The optimization problem is solved by Genetic Algorithm (GA) considering both linear and nonlinear equality and inequality constraints. Due to the very large number of candidate sets available for sensor placement and because of the long time that it takes to solve the constrained optimization problem that includes more than 1000 states, solution of this problem is computationally expensive. For reducing the computation time, parallel computing is performed using the Distributed Computing Server (DCS®) and the Parallel Computing® toolbox from Mathworks®. In this presentation, we will share our experience in setting up parallel computing using GA in the MATLAB® environment and present the overall approach for achieving higher computational efficiency in this framework.

97 MATHEMATICS AND COMPUTING↗

Sensor placement algorithm development to maximize the efficiency of acid gas removal unit for integrated gasification combined cycle (IGCC) power plant with CO{sub 2} capture

Future integrated gasification combined cycle (IGCC) power plants with CO{sub 2} capture will face stricter operational and environmental constraints. Accurate values of relevant states/outputs/disturbances are needed to satisfy these constraints and to maximize the operational efficiency. Unfortunately, a number of these process variables cannot be measured while a number of them can be measured, but have low precision, reliability, or signal-to-noise ratio. In this work, a sensor placement (SP) algorithm is developed for optimal selection of sensor location, number, and type that can maximize the plant efficiency and result in a desired precision of the relevant measured/unmeasured states. In this work, an SP algorithm is developed for an selective, dual-stage Selexol-based acid gas removal (AGR) unit for an IGCC plant with pre-combustion CO{sub 2} capture. A comprehensive nonlinear dynamic model of the AGR unit is developed in Aspen Plus Dynamics® (APD) and used to generate a linear state-space model that is used in the SP algorithm. The SP algorithm is developed with the assumption that an optimal Kalman filter will be implemented in the plant for state and disturbance estimation. The algorithm is developed assuming steady-state Kalman filtering and steady-state operation of the plant. The control system is considered to operate based on the estimated states and thereby, captures the effects of the SP algorithm on the overall plant efficiency. The optimization problem is solved by Genetic Algorithm (GA) considering both linear and nonlinear equality and inequality constraints. Due to the very large number of candidate sets available for sensor placement and because of the long time that it takes to solve the constrained optimization problem that includes more than 1000 states, solution of this problem is computationally expensive. For reducing the computation time, parallel computing is performed using the Distributed Computing Server (DCS®) and the Parallel Computing® toolbox from Mathworks®. In this presentation, we will share our experience in setting up parallel computing using GA in the MATLAB® environment and present the overall approach for achieving higher computational efficiency in this framework.

20 FOSSIL-FUELED POWER PLANTS↗

SSCA1-K1

SSCA1-K1 is a parallel implementation of kernel 1 of the SSCA1 benchmark suite released by the DARPA HPCS program. This kernel is able to run in parallel on a distributed shared memory system at extreme scales using OpenSHMEM.

Baker, Matthew↗

OpenSHMEM over MPI-3

OpenSHMEM is a community effort to unifyt and standardize the SHMEM programming model. MPI (Message Passing Interface) is a well-known community standard for parallel programming using distributed memory. The most recen t release of MPI, version 3.0, was designed in part to support programming models like SHMEM.OSHMPI is an implementation of the OpenSHMEM standard using MPI-3 for the Linux operating system. It is the first implementation of SHMEM over MPI one-sided communication and has the potential to be widely adopted due to the portability and widely availability of Linux and MPI-3. OSHMPI has been tested on a variety of systems and implementations of MPI-3, includingInfiniBand clusters using MVAPICH2 and SGI shared-memory supercomputers using MPICH. Current support is limited to Linux but may be extended to Apple OSX if there is sufficient interest. The code is opensource via https://github.com/jeffhammond/oshmpi

Sayan Ghosh, Jeff Hammond↗