Search NASA⌕ Search

SEARCH · Search NASA

Results for “Computational Load Balancing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Load‐Balancing Intense Physics Calculations to Embed Regionalized High‐Resolution Cloud Resolving Models in the E3SM and CESM Climate Models

Abstract We design a new strategy to load‐balance high‐intensity sub‐grid atmospheric physics calculations restricted to a small fraction of a global climate simulation's domain. We show why the current parallel load balancing infrastructure of Community Earth System Model (CESM) and Energy Exascale Earth Model (E3SM) cannot efficiently handle this scenario at large core counts. As an example, we study an unusual configuration of the E3SM Multiscale Modeling Framework (MMF) that embeds a binary mixture of two separate cloud‐resolving model grid structures that is attractive for low cloud feedback studies. Less than a third of the planet uses high‐resolution (MMF‐HR; sub‐km horizontal grid spacing) relative to standard low‐resolution (MMF‐LR) cloud superparameterization elsewhere. To enable MMF runs with Multi‐Domain cloud resolving models (CRMs), our load balancing theory predicts the most efficient computational scale as a function of the high‐intensity work's relative overhead and its fractional coverage. The scheme successfully maximizes model throughput and minimizes model cost relative to precursor infrastructure, effectively by devoting the vast majority of the processor pool to operate on the few high‐intensity (and rate‐limiting) high‐resolution (HR) grid columns. Two examples prove the concept, showing that minor artifacts can be introduced near the HR/low‐resolution CRM grid transition boundary on idealized aquaplanets, but are minimal in operationally relevant real‐geography settings. As intended, within the high (low) resolution area, our Multi‐Domain CRM simulations exhibit cloud fraction and shortwave reflection convergent to standard baseline tests that use globally homogenous MMF‐LR and MMF‐HR. We suggest this approach can open up a range of creative multi‐resolution climate experiments without requiring unduly large allocations of computational resources.

54 ENVIRONMENTAL SCIENCES↗

Asynchronous Navier-Stokes Solver on 3D Unstructured Grids for the Exascale Era

This project has developed multiple fluid dynamics solvers for complex 3D flows using fully asynchronous distributed-memory task-parallel algorithms on top of the Charm++ runtime system. The algorithms solve the Euler or Navier-Stokes equations of compressible flows using unstructured tetrahedron meshes with optional solution-adaptive mesh-, and polynomial-order refinement. We have demonstrated excellent strong scaling up 50K compute cores and the benefits of Charm++’s automatic load balancing.

97 MATHEMATICS AND COMPUTING↗

True Load Balancing for Matricized Tensor Times Khatri-Rao Product

MTTKRP is the bottleneck operation in algorithms used to compute the CP tensor decomposition. For sparse tensors, utilizing the compressed sparse fibers (CSF) storage format and the CSF-oriented MTTKRP algorithms is important for both memory and computational efficiency on distributed-memory architectures. Existing intelligent tensor partitioning models assume the computational cost of MTTKRP to be proportional to the total number of nonzeros in the tensor. However, this is not the case for the CSF-oriented MTTKRP on distributed-memory architectures. We outline two deficiencies of nonzero-based intelligent partitioning models when CSF-oriented MTTKRP operations are performed locally: failure to encode processors' computational loads and increase in total computation due to fiber fragmentation. We focus on existing fine-grain hypergraph model and propose a novel vertex weighting scheme that enables this model encode correct computational loads of processors. We also propose to augment the fine-grain model by fiber nets for reducing the increase in total computational load via minimizing fiber fragmentation. In this way, the proposed model encodes minimizing the load of the bottleneck processor. In conclusion, parallel experiments with real-world sparse tensors on up to 1024 processors prove the validity of the outlined deficiencies and demonstrate the merit of our proposed improvements in terms of parallel runtimes.

97 MATHEMATICS AND COMPUTING↗

PLATO (Parallel Load Assignment Tool): A Parallel Workload Partitioner for Particle-based Methods

A key first step in many large-scale high-performance scientific computing applications is the distribution of the computational domain across all participating processors of a parallel computing platform such as a shared memory machine, cluster or a tightly-coupled supercomputer. The primary goal of such a partitioning step is to ensure that the computational work performed by each process remains load-balanced throughout the execution of the parallel application. In this project, a general-purpose parallel partitioning tool, code-named PLATO, was designed and implemented. PLATO assumes that the workload is a set of points in an n-dimensional metric space. These points are abstractions of computational units that may represent point-like computational units such as celestial objects in astrophysics, atoms in materials sciences and molecules in biology/chemistry, or may represent more abstract computational units such as grid points in finite difference methods, grid elements in finite element methods or n-dimensional points in the feature space of a host of other applications. PLATO supports several key and useful capabilities, namely: (a) multiple load partitioning strategies, (b) support for arbitrary user-defined processor topology, (c) pre-fetching data from neighboring processors based on user-defined parameters, and (d) support for arbitrary number and types of attributes associated with each point in the computational domain. PLATO is implemented in the C programming language using Message Passing Interface (MPI) for inter-processor communications and its parallel performance tested on up to 48 processors of a shared memory machine. This report includes a detailed performance study of PLATO using parallel speedup and iso-efficiency as the two main metrics of parallel performance. In addition, a user manual and DOXYGEN generated code documentation have been included for completeness.

97 MATHEMATICS AND COMPUTING↗

Applying Graph Partitioning Methods in Measurement-Based Dynamic Load Balancing

Load imbalance can lead to wasted CPU hours especially when running on a large number of processors. Achieving the best parallel efficiency for a program requires optimal load balancing which is a NP-hard problem. Charm++, a migratable objects based programming model, provides a measurement-based dynamic load balancing frame work. This paper explores the use of graph partitioning algorithms, traditionally used for partitioning physical domains/meshes, for measurement based dynamic load balancing of parallel applications. In particular, we present repartitioning methods developed in a graph partitioning toolbox called Scotch that consider the previous mapping to minimize migration costs. We also discuss a new imbalance reduction algorithm for graphs with irregular load distributions. We compare several load balancing algorithms using a micro-benchmark and the NAS BT multi-zone bench mark. New algorithms developed in Scotch lead to better performance compared to other existing partitioners, both in terms of the application execution time and fewer number of objects migrated.

97 MATHEMATICS AND COMPUTING↗

Task-based Parallel Computation of the Density Matrix in Quantum-based Molecular Dynamics using Graph Partitioning

Quantum-based molecular dynamics (QMD) is a highly accurate and transferable method for material science simulations. However, the time scales and system sizes accessible to QMD are typically limited to picoseconds and a few hundred atoms. These constraints arise due to expensive self-consistent ground-state electronic structure calculations that can often scale cubically with the number of atoms. Linearly scaling methods depend on computing the density matrix P from the Hamiltonian matrix H by exploiting the sparsity in both matrices. The second-order spectral projection (SP2) algorithm is an O(N) algorithm that computes P with a sequence of 40 - 50 matrix-matrix multiplications. In this paper, we present task-based implementations of a recently developed data-parallel graph-based approach to the SP2 algorithm, G-SP2. We represent the density matrix P as an undirected graph and use graph partitioning techniques to divide the computation into smaller independent tasks. The partitions thus obtained are generally not of equal size and give rise to undesirable load imbalances in standard MPI-based implementations. This load-balancing challenge can be mitigated by dynamically scheduling parallel computations at runtime using task-based programming models. We develop task-based implementations of the data-parallel G-SP2 algorithm using both Intel's Concurrent Collections (CnC) as well as the Charm++ programming model and evaluate these implementations for future use. Scaling and performance results of our implementations are investigated for representative segments of QMD simulations for solvated protein systems containing more than 10,000 atoms.

97 MATHEMATICS AND COMPUTING↗

Computing Properties of Hadrons, Nuclei and Nuclear Matter from Quantum Chromodynamics

This project was part of a coordinated software development effort which the nuclear physics lattice QCD community pursues in order to ensure that lattice calculations can make optimal use of present, and forthcoming leadership-class and dedicated hardware, including those of the national laboratories, and prepares for the exploitation of future computational resources in the exascale era. The UW team improved and extended software libraries used in lattice QCD calculations related to multi-nucleon systems, enhanced production running codes related to load balancing multi-nucleon production on large-scale computing platforms, and developed SQLite (addressable database) interfaces to efficiently archive and analyze multi-nucleon data and developed a Mathematica interface for the SQLite databases.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Planning for Variable Generation Integration through Balancing Authorities Consolidation

As more and more variable generation is integrated into power grids, many challenges and concerns arise for an individual balancing authority (BA) to balance the system with limited resources. Consolidating balancing authorities provides a promising method to mitigate these problems by enabling the sharing of resources through operating different BAs as a single BA. The diversity in load and renewable generation over a wide area can be effectively leveraged, which makes it possible to achieve significant savings in balancing requirements. This paper develops a detailed procedure to compute savings in load following and regulation service requirements due to BAs consolidation. It proposes several evaluation metrics for demonstrating the benefits of BA consolidation. Several study scenarios are designed for a set of BAs in the western United States to test the proposed procedure. Results have shown significant savings in the capacity, ramp rate, and energy of balancing service requirements. Important factors affecting the savings, such as forecast accuracy and cross correlation between forecast errors, are also discussed.

Diao, Ruisheng↗

On Design of a Coupling Component for Parallel Multimodeling

A component-based simulation framework is a favorable choice for many multi-scale simulations, which requires dedicated coupling to support flexible data communication/exchange between components, computational parallelism and on-demand dynamic load balancing. In this paper, authors first present the considerations in the course of design and implementation of a coupling component to support integrated parallel multimodeling. Then, authors use an integrated ecological modeling case, as a part of the Across Trophic Level System Simulation (ATLSS) for the ecological restoration projects in the Everglades of South Florida, to demonstrate the practical application of such component in real-world simulations. In the case study, we illustrated the internal data flow between the ecological models and the performance enhancement of the integrated simulation via the coupling component.

99 GENERAL AND MISCELLANEOUS↗

Communication Simulations for Power System Applications

New smart grid technologies and concepts, such as dynamic pricing, demand response, dynamic state estimation, and wide area monitoring, protection, and control, are expected to require considerable communication resources. As the cost of retrofit can be high, future power grids will require the integration of high-speed, secure connections with legacy communication systems, while still providing adequate system control and security. While considerable work has been performed to create co-simulators for the power domain with load models and market operations, limited work has been performed in integrating communications directly into a power domain solver. The simulation of communication and power systems will become more important as the two systems become more inter-related. This paper will discuss ongoing work at Pacific Northwest National Laboratory to create a flexible, high-speed power and communication system co-simulator for smart grid applications. The framework for the software will be described, including architecture considerations for modular, high performance computing and large-scale scalability (serialization, load balancing, partitioning, cross-platform support, etc.). The current simulator supports the ns-3 (telecommunications) and GridLAB-D (distribution systems) simulators. Ongoing and future work will be described, including planned future expansions for a traditional transmission solver. A test case using the co-simulator, utilizing a transactive demand response system created for the Olympic Peninsula and AEP gridSMART demonstrations, requiring two-way communication between distributed and centralized market devices, will be used to demonstrate the value and intended purpose of the co-simulation environment.

co-simulation↗

Efficient Support for Matrix Computations on Heterogeneous Multi-core and Multi-GPU Architectures

We present a new methodology for utilizing all CPU cores and all GPUs on a heterogeneous multicore and multi-GPU system to support matrix computations e ciently. Our approach is able to achieve the objectives of a high degree of parallelism, minimized synchronization, minimized communication, and load balancing. Our main idea is to treat the heterogeneous system as a distributed-memory machine, and to use a heterogeneous 1-D block cyclic distribution to allocate data to the host system and GPUs to minimize communication. We have designed heterogeneous algorithms with two di erent tile sizes (one for CPU cores and the other for GPUs) to cope with processor heterogeneity. We propose an auto-tuning method to determine the best tile sizes to attain both high performance and load balancing. We have also implemented a new runtime system and applied it to the Cholesky and QR factorizations. Our experiments on a compute node with two Intel Westmere hexa-core CPUs and three Nvidia Fermi GPUs demonstrate good weak scalability, strong scalability, load balance, and e ciency of our approach.

97 MATHEMATICS AND COMPUTING↗

Leveraging In-Network Computing and Programmable Switches for Streaming Analysis of Scientific Data

With the emergence of programmable network devices that match the performance of fixed function devices, several recent projects have explored in-network computing, where the processing that is traditionally done outside the network is offloaded to the network devices. In-network computing has typically been applied to network functions (e.g., load balancing, NAT, and DNS), caching, data reduction/aggregation, and coordination/consensus functions. In some cases it has been used to accelerate stream-processing tasks that involve small payloads and simple operations. In this work we focus on leveraging in-network computing for stream processing of scientific datasets with large payloads that require complex operations such as floating-point computations and logarithmic functions. We demonstrate in-network computing for a real-world scientific application performing streaming normalization of a 2-D image from a light source experiment. We discuss the challenges we encountered and potential approaches to address them.

Sankaran, Ganesh↗

An accelerated approach to inline non-LTE modeling

A goal across many physics and engineering disciplines is to use radiation-hydrodynamics codes to accurately simulate systems that are expected to deviate from local thermodynamic equilibrium (LTE), such as laser-driven hohlraums, stellar coronae, and supernova ejecta. The computational expense of traditional non-LTE (NLTE) opacity calculations can dominate the cost of simulating these systems and render NLTE physics untenable in most cases. In this paper, we present a new parallel implementation for NLTE opacity calculations that employs advanced load balancing, vectorization, and memory management to significantly reduce the computational costs of these calculations. Using the radiation-hydrodynamics code xRAGE , we demonstrate this new implementation on three test problems and present order-of-magnitude speed improvements.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Development of Parallel Computing Framework to Enhance Radiation Transport Code Capabilities for Rare Isotope Beam Facility Design

A parallel computing framework has been developed to use with general-purpose radiation transport codes. The framework was implemented as a C++ module that uses MPI for message passing. It is intended to be used with older radiation transport codes implemented in Fortran77, Fortran 90 or C. The module is significantly independent of radiation transport codes it can be used with, and is connected to the codes by means of a number of interface functions. The framework was developed and tested in conjunction with the MARS15 code. It is possible to use it with other codes such as PHITS, FLUKA and MCNP after certain adjustments. Besides the parallel computing functionality, the framework offers a checkpoint facility that allows restarting calculations with a saved checkpoint file. The checkpoint facility can be used in single process calculations as well as in the parallel regime. The framework corrects some of the known problems with the scheduling and load balancing found in the original implementations of the parallel computing functionality in MARS15 and PHITS. The framework can be used efficiently on homogeneous systems and networks of workstations, where the interference from the other users is possible.

43 PARTICLE ACCELERATORS↗

Dynamic load balancing algorithm for molecular dynamics based on Voronoi cells domain decompositions

We present a new algorithm for automatic parallel load balancing in classical molecular dynamics. It assumes a spatial domain decomposition of particles into Voronoi cells. It is a gradient method which attempts to minimize a cost function by displacing Voronoi sites associated with each processor/sub-domain along steepest descent directions. Excellent load balance has been obtained for quasi-2D and 3D practical applications, with up to 440·10 6 particles on 65,536 MPI tasks.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

A Framework for Load Balancing of Tensor Contraction Expressions via Dynamic Task Partitioning

In this paper, we introduce the Dynamic Load-balanced Tensor Contractions (DLTC), a domain-specific library for efficient task parallel execution of tensor contraction expressions, a class of computation encountered in quantum chemistry and physics. Our framework decomposes each contraction into smaller unit of tasks, represented by an abstraction referred to as iterators. We exploit an extra level of parallelism by having tasks across independent contractions executed concurrently through a dynamic load balancing run- time. We demonstrate the improved performance, scalability, and flexibility for the computation of tensor contraction expressions on parallel computers using examples from coupled cluster methods.

Tensor contraction, domain-specific language, dyna↗

Galactos: computing the anisotropic 3-point correlation function for 2 billion galaxies

The nature of dark energy and the complete theory of gravity are two central questions currently facing cosmology. A vital tool for addressing them is the 3-point correlation function (3PCF), which probes deviations from a spatially random distribution of galaxies. However, the 3PCF's formidable computational expense has prevented its application to astronomical surveys comprising millions to billions of galaxies. We present Galactos, a high-performance implementation of a novel, O( N 2 ) algorithm that uses a load-balanced k-d tree and spherical harmonic expansions to compute the anisotropic 3PCF. Our implementation is optimized for the Intel Xeon Phi architecture, exploiting SIMD parallelism, instruction and thread concurrency, and significant L1 and L2 cache reuse, reaching 39% of peak performance on a single node. Galactos scales to the full Cori system, achieving 9.8 PF (peak) and 5.06 PF (sustained) across 9636 nodes, making the 3PCF easily computable for all galaxies in the observable universe.

79 ASTRONOMY AND ASTROPHYSICS↗

New Parallel computing framework for radiation transport codes

A new parallel computing framework has been developed to use with general-purpose radiation transport codes. The framework was implemented as a C++ module that uses MPI for message passing. The module is significantly independent of radiation transport codes it can be used with, and is connected to the codes by means of a number of interface functions. The framework was integrated with the MARS15 code, and an effort is under way to deploy it in PHITS. Besides the parallel computing functionality, the framework offers a checkpoint facility that allows restarting calculations with a saved checkpoint file. The checkpoint facility can be used in single process calculations as well as in the parallel regime. Several checkpoint files can be merged into one thus combining results of several calculations. The framework also corrects some of the known problems with the scheduling and load balancing found in the original implementations of the parallel computing functionality in MARS15 and PHITS. The framework can be used efficiently on homogeneous systems and networks of workstations, where the interference from the other users is possible.

43 PARTICLE ACCELERATORS↗