Search NASA⌕ Search

SEARCH · Search NASA

Results for “high performance and distributed computing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,387 records · Page 77

Segmentation of RDX and TNT in X‐Ray Computed Tomography Reconstructions of Melt‐Cast Explosives

ABSTRACT Three‐dimensional mesoscale characterization of heterogeneous melt‐cast high explosives is challenging because of the difficulty differentiating binder from explosive crystals: two functionally different materials which are typically similar in density by design. Here, we report an algorithm which can differentiate hexahydro‐1,3,5‐trinitro‐1,3,5‐triazine (RDX) from 2,4,6‐trinitrotoluene (TNT) in x‐ray computed tomography (CT) volumes with tens of microns resolution. This method allows us to quantify RDX/TNT content, porosity, and RDX domain size. We calibrated the segmentation algorithm using simulated x‐ray CT volumes containing object models of RDX crystals within a TNT matrix. We then segmented and analyzed CT data for Composition B (Comp B), a 60/40 RDX/TNT mixture, and Cyclotol, a 75/25 RDX/TNT mixture. We examined melt‐cast samples fabricated with 100% theoretical maximum density (TMD) and 85% TMD. For the 100% TMD Comp B and Cyclotol samples, the RDX content values calculated by segmentation were 3% and 9% lower, respectively, than the values measured by high‐performance liquid chromatography on material from the same synthesis lots. This result is consistent with the expected underreporting of RDX content resulting from x‐ray CT resolution limits on RDX particles with diameters smaller than 25 µm. The 85% TMD samples were less accurately segmented with our algorithm due to the confounding presence of voids.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Development of predictive capability of cycle-to-cycle variation in dual-fuel engines using supercomputing-based computational fluid dynamics

Cycle-to-cycle variation (CCV) in internal combustion engines is expected due to variabilities in operating conditions such as in-cylinder fuel stratification, mixing behavior, manifold air pressure, injection timing etc. Generally, engine experimental studies provide averaged quantities such as cylinder pressure, emissions, indicated mean effective pressure (IMEP) from multiple cycles and their CCV of IMEP, and CCV of cylinder pressure from consecutive cycles for a given test condition. Capturing 3D spatial distributions of relevant measures such as fuel-air mixing, temperature, turbulence levels and emissions from such experiments is challenging. Computational Fluid Dynamics (CFD) is an alternative to experiments, that can be used effectively to understand the spatial and temporal distributions of parameters of interest.Quantitative understanding of CCV in dual-fuel (DF) engines will accelerate the development and deployment of engines that provide fuel flexibility, economic advantage and emission compliance. Earlier work [1] performed to develop an understanding of the physics of CCV in large-bore, medium-speed, dual-fuel engines was partially successful. That effort entailed a sparse design of experiment (DOE) of full-cylinder geometry, closed-cycle, dual-fuel CFD simulations in which variabilities within some of the identified global parameters were exercised. The Coefficient of Variation (COV) of cylinder pressure was promising compared with experimental COV. This reinforced the understanding of which global parameters are significant contributors to CCV in DF combustion.The authors have used several high-performance supercomputing facilities in the process of developing predicting capability of CCV in large-bore engines, while operating in dual-fuel, diesel and natural gas (NG), combustion mode [1, 2]. In the current study CCV influencing parameters in a diesel-natural gas dual-fuel engine was studied using a 4th order DOE consisting of 137 concurrent cases. These full-geometry, closed-cycle, CFD simulations were run on the Mira supercomputing facility at the Argonne National Laboratory (ANL). This paper describes key learnings of that study along with a validation of the DOE. Further, the CCV prediction methodology, which is applicable to simulation-based design and testing, is described.

Gubba, Sreenivasa Rao↗

On the influence of over-parameterization in manifold based surrogates and deep neural operators

Constructing accurate and generalizable approximators (surrogate models) for complex physico-chemical processes exhibiting highly non-smooth dynamics is challenging. The main question is what type of surrogate models we should construct and should these models be under-parameterized or over-parameterized. In this work, we propose new developments and perform comparisons for two promising approaches: manifold-based polynomial chaos expansion (m-PCE) and the deep neural operator (DeepONet), and we examine the effect of over-parameterization on generalization. While m-PCE enables the construction of a mapping by first identifying low-dimensional embeddings of the input functions, parameters, and quantities of interest (QoIs), a neural operator learns the nonlinear mapping via the use of deep neural networks. Here, we demonstrate the performance of these methods in terms of generalization accuracy by solving the 2D time-dependent Brusselator reaction-diffusion system with uncertainty sources, modeling an autocatalytic chemical reaction between two species. We first propose an extension of the m-PCE by constructing a mapping between latent spaces formed by two separate embeddings of the input functions and the output QoIs. To further enhance the accuracy of the DeepONet, we introduce weight self-adaptivity in the loss function. We demonstrate that the performance of m-PCE and DeepONet is comparable for cases of relatively smooth input-output mappings. However, when highly non-smooth dynamics is considered, DeepONet shows higher approximation accuracy. We also find that for m-PCE, modest over-parameterization leads to better generalization, both within and outside of distribution, whereas aggressive over-parameterization leads to over-fitting. In contrast, an even highly over-parameterized DeepONet leads to better generalization for both smooth and non-smooth dynamics. Furthermore, we compare the performance of the above models with another recently proposed operator learning model, the Fourier Neural Operator, and show that its over-parameterization also leads to better generalization. Taken together, our studies show that m-PCE can provide very good accuracy at very low training cost, whereas a highly over-parameterized DeepONet can provide better accuracy and robustness to noise but at higher training cost. In both methods, the inference cost is negligible.

97 MATHEMATICS AND COMPUTING↗

An Open-Source Parallel EMT Simulation Framework: Preprint

As the integration level of inverter-based resources (IBR) increases, ensuring the reliable operation of the bulk power systems requires the use of electromagnetic transient (EMT) simulation tools to identify and mitigate system-wide stability risks. Conducting EMT studies for large-scale, IBR-rich grids, however, is challenging due to the inherent computational bottleneck caused by the underlying high-fidelity models and required small time steps. This paper introduces ParaEMT: an open-source, generic EMT simulation framework designed to accelerate simulations by leveraging advanced parallel computational technologies, such as high-performance computers. This paper presents a comprehensive exposition of ParaEMT, covering its modeling library, simulation strategy, framework structure, operational procedures, and auxiliary features, alongside its extensible parallel computational architecture. Notably, ParaEMT is a publicly accessible and modularized framework written in Python, thereby facilitating future development and the integration of new models and algorithms. The accuracy and efficiency of ParaEMT are demonstrated by rigorous validations via multiple case studies.

electromagnetic transient simulation↗

Let’s get our hands dirty: a comprehensive evaluation of DAQDB, key-value store for petascale hot storage

Data acquisition systems are a key component for successful data taking in any experiment. The DAQ is a complex distributed computing system and coordinates all operations, from the data selection stage of interesting events to storage elements. For the High Luminosity upgrade of the Large Hadron Collider, the experiments at CERN need to meet challenging requirements to record data with a much higher occupancy in the detectors. The DAQ system will receive and deliver data with a significantly increased trigger rate, one million events per second, and capacity, terabytes of data per second. An effective way to meet these requirements is to decouple real-time data acquisition from event selection. Data fragments can be temporarily stored in a large distributed key-value store. Fragments belonging to the same event can be then queried on demand, by the data selection processes. Implementing such a model relies on a proper combination of emerging technologies, such as persistent memory, NVMe SSDs, scalable networking, and data structures, as well as high performance, scalable software. In this paper, we present DAQDB (Data Acquisition Database) — an open source implementation of this design that was presented earlier, with an extensive evaluation of this approach, from the single node to the distributed performance. Furthermore, we complement our study with a description of the challenges faced and the lessons learned while integrating DAQDB with the existing software framework of the ATLAS experiment.

Abed Abud, Adam↗

WLCG and IPv6 - The HEPiX IPv6 working group

The HEPiX (http://www.hepix.org) IPv6 Working Group has been investigating the many issues which feed into the decision on the timetable for the use of IPv6 (http://www.ietf.org/rfc/rfc2460.txt) networking protocols in High Energy Physics (HEP) Computing, in particular in the Worldwide Large Hadron Collider (LHC) Computing Grid (WLCG). RIPE NCC, the European Regional Internet Registry (RIR), ran out ofIPv4 addresses in September 2012. The North and South America RIRs are expected to run out soon. In recent months it has become more clear that some WLCG sites, including CERN, are running short of IPv4 address space, now without the possibility of applying for more. This has increased the urgency for the switch-on of dual-stack IPv4/IPv6 on all outward facing WLCG services to allow for the eventual support of IPv6-only clients. The activities of the group include the analysis and testing of the readiness for IPv6 and the performance of many required components, including the applications, middleware, management and monitoring tools essential for HEP computing. Many WLCG Tier 1/2 sites are participants in the group's distributed IPv6 testbed and the major LHC experiment collaborations are engaged in the testing. We are constructing a group web/wiki which will contain useful information on the IPv6 readiness of the various software components and a knowledge base (http://hepix-ipv6.web.cern.ch/knowledge-base). Furthermore, this paper describes the work done by the working group and its future plans.

43 PARTICLE ACCELERATORS↗

Contactless Position Measurement System for Remote Alignment of Highly Reflective Objects

The Contactless Position Measurement System (CPMS) permits automatic alignment of highly-reflecting components that cannot or should not be touched by human hands, such as those used in SRF cavities. CPMS also has application during maintenance of fusion reactors, when the components in the reactor are radioactive. The length of time for which humans can handle the components in order to carry out maintenance or repairs is limited. We would like to measure the relative positions of such objects so we can bring them into alignment with motorized stages, thus eliminating the need for human contact. The CPMS is a computer vision system that measures the location of highly reflective components to within a fraction of a millimeter by finding the edges of silhouette images. This is done by placing a uniform, infrared backlight behind the components, and viewing each component with two low-aberration, infrared cameras. Each of these stereoscopic cameras are located within a coordinate system we set up with reference platforms, light sources, and survey cameras distributed around the perimeter of the string assembly room, or fusion reactor. We obtain stereoscopic silhouette images of each component, and we use these to determine the position of the component within our string assembly coordinate system. The CPMS combines the silhouette images with knowledge of the dimensions of the objects to obtain the relative positions of components in its field of view. These position measurements then allow us to mechanically maneuver the components into contact. Once we know where they are, we can move the components to where they are supposed to be with motorized stages or other mechanical means, check they are in the right place, and bolt them together. In Phase I, we built a prototype infrared backlights and cameras, tested it, wrote an analysis program to fit the images. At a range of 50 cm, we are able to measure the relative positions of two stainless steel flanges with an accuracy of 150 μm rms. The next steps are to test the analysis program with motorized stages, build a full-size prototype, and implement it in an cleanroom with SRF cavity assembly. This can be done in a cost effective way given the materials with which the system is designed. As Open Source Instruments achieved more in Phase I than anticipated, despite not receiving a Phase II SBIR grant, we hope to move forward with product development and sales with a testing partner.

43 PARTICLE ACCELERATORS↗

Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Bayesian inference↗

Magnonic band structure, complete bandgap, and collective spin wave excitation in nanoscale two-dimensional magnonic crystals

We present the observation of a complete bandgap and collective spin wave excitation in two-dimensional magnonic crystals comprised of arrays of nanoscale antidots and nanodots, respectively. Considering that the frequencies dealt with here fall in the microwave band, these findings can be used for the development of suitable magnonic metamaterials and spin wave based signal processing. We also present the application of a numerical procedure, to compute the dispersion relations of spin waves for any high symmetry direction in the first Brillouin zone. The results obtained from this procedure have been reproduced and verified by the well established plane wave method for an antidot lattice, when magnetization dynamics at antidot boundaries are pinned. The micromagnetic simulation based method can also be used to obtain iso–frequency contours of spin waves. Iso–frequency contours are analogous of the Fermi surfaces and hence, they have the potential to radicalize our understanding of spin wave dynamics. The physical origin of bands, partial and full magnonic bandgaps have been explained by plotting the spatial distribution of spin wave energy spectral density. Although, unfettered by rigid assumptions and approximations, which afflict most analytical methods used in the study of spin wave dynamics, micromagnetic simulations tend to be computationally demanding. Thus, the observation of collective spin wave excitation in the case of nanodot arrays, which can obviate the need to perform simulations, may also prove to be valuable.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Effectively Using Remote I/O For Work Composition in Distributed Workflows

Distributed scientific workflows are becoming more important with the interest in incorporating AI into their loops. A critical programming and performance question is how to compose workflow tasks when data is produced on one system but must be consumed on another. Since the dominant technique is composition with remote I/O, this paper explores its performance expectations. We describe BigFlowSim, a workflow I/O simulator that captures key implementation choices for remote I/O, including intensity, reuse, locality, access pattern, and data movement.With BigFlowSim, we generate a synthetic benchmark. We quantify the effects of each parameter with a performance sensitivity study. We explain trends in terms of data movement reduction and show that, under certain conditions, it is possible to establish a total order among most parameters. We apply these insights to a high energy physics workflow, Belle II Monte Carlo and simulate several I/O optimizations. Speedups range from 5% to 2×, without changing compute time.

Friese, Ryan D.↗

Dosimetric Impact of Interplay Effect on RapidArc Lung Stereotactic Treatment Delivery

Purpose: Volumetric modulated arc therapy (RapidArc; Varian Medical Systems, Palo Alto, CA) allows fast delivery of stereotactic radiotherapy for Stage I lung tumors. We investigated discrepancies between the calculated and delivered dose distributions, as well as the dosimetric impact of leaf interplay with breathing-induced tumor motion. Methods and Materials: In 20 consecutive patients with Stage I lung cancer who completed RapidArc delivery, 15 had tumor motion exceeding 5 mm on four-dimensional computed tomography scan. Static and dynamic measurements were performed with Gafchromic EBT film (International Specialty Products Inc., Wayne, NJ) in a Quasar motion phantom (Modus Medical Devices, London, Ontario, Canada). Static measurements were compared with calculated dose distributions, and dynamic measurements were compared with the convolution of static measurements with sinusoidal motion patterns. Besides clinical treatment plans, additional cases were optimized to create excessive multileaf collimator modulation and delivered on the phantom with peak-to-peak motions of up to 25 mm. {gamma} Analysis with a 3% dose difference and 2- or 1-mm distance to agreement was used to evaluate the accuracy of delivery and the dosimetric impact of the interplay effect. Results: In static mode film dosimetry of the two-arc delivery in the phantom showed that, on average, fewer than 3% of measurements had {gamma} greater than 1. Dynamic measurements of clinical plans showed a high degree of agreement with the convolutions: for double-arc plans, 99.5% met the {gamma} criterion. The degree of agreement was 98.5% for the plans with excessive multileaf collimator modulations and 25 mm of motion. Conclusions: Film dosimetry shows that RapidArc accurately delivers the calculated dose distribution and that interplay between leaves and tumor motion is not significant for single-fraction treatments when RapidArc is delivered with two different arcs.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Epidemic failure detection and consensus for extreme parallelism

Future extreme-scale high-performance computing systems will be required to work under frequent component failures. The MPI Forum s User Level Failure Mitigation proposal has introduced an operation, MPI Comm shrink, to synchronize the alive processes on the list of failed processes, so that applications can continue to execute even in the presence of failures by adopting algorithm-based fault tolerance techniques. This MPI Comm shrink operation requires a failure detection and consensus algorithm. This paper presents three novel failure detection and consensus algorithms using Gossiping. The proposed algorithms were implemented and tested using the Extreme-scale Simulator. The results show that in all algorithms the number of Gossip cycles to achieve global consensus scales logarithmically with system size. The second algorithm also shows better scalability in terms of memory and network bandwidth usage and a perfect synchronization in achieving global consensus. The third approach is a three-phase distributed failure detection and consensus algorithm and provides consistency guarantees even in very large and extreme-scale systems while at the same time being memory and bandwidth efficient.

97 MATHEMATICS AND COMPUTING↗

Evaluating Emulation-based Models of Distributed Computing Systems

Emulation-based models of distributed computing systems are collections of virtual ma- chines, virtual networks, and other emulation components configured to stand in for oper- ational systems when performing experimental science, training, analysis of design alterna- tives, test and evaluation, or idea generation. As with any tool, we should carefully evaluate whether our uses of emulation-based models are appropriate and justified. Otherwise, we run the risk of using a model incorrectly and creating meaningless results. The variety of uses of emulation-based models each have their own goals and deserve thoughtful evaluation. In this paper, we enumerate some of these uses and describe approaches that one can take to build an evidence-based case that a use of an emulation-based model is credible. Predictive uses of emulation-based models, where we expect a model to tell us something true about the real world, set the bar especially high and the principal evaluation method, called validation , is comensurately rigorous. We spend the majority of our time describing and demonstrating the validation of a simple predictive model using a well-established methodology inherited from decades of development in the compuational science and engineering community.

97 MATHEMATICS AND COMPUTING↗

Leveraging FPGA Advantages for Quicker Data Processing for LBNF

The Long Baseline Neutrino Facility (LBNF) will deliver a 2.4 MW muon neutrino beam from Fermilab to the Deep Underground Neutrino Experiment (DUNE), requiring unprecedented precision in beamline alignment to achieve DUNE's neutrino oscillation measurement goals. Vertical misalignments of beamline components as small as 0.5 mm can contribute 6-7\% uncertainty in predicted neutrino flux, necessitating sub-0.1 mm alignment monitoring capabilities. The Horn Location Sensor (HLS) system employs frequency sweep interferometry (FSI) in a distributed hydrostatic leveling network to achieve the required precision under harsh radiation conditions up to 5000 kRad/year. Traditional FSI implementations suffer from laser sweep nonlinearities that degrade resolution and require computationally intensive post-processing corrections using gas reference cells. This work presents a real-time FPGA-based implementation of the HLS data acquisition and processing system using a sweep tracker interferometer for dynamic sweep linearization. The system utilizes a PYNQ-Z2 FPGA with programmable logic implementing parallel 16k-point FFT processing across four channels, synchronized by the sweep tracker signal to eliminate post-processing requirements. Spectral performance testing demonstrates significant improvements in peak sharpness compared to traditional fixed-frequency digitization. The FPGA implementation enables real-time displacement monitoring with processing speeds orders of magnitude faster than software-based approaches, essential for the operational requirements of LBNF's eventual distributed sensor network. This advancement in real-time FSI processing directly supports DUNE's precision neutrino physics program by providing the rapid feedback necessary for maintaining stringent beamline alignment tolerances during high-power beam operations.

Rossel, Jacob↗

Real-Time FPGA Implementation For Frequency Sweep Interferometry In The LBNF Complex

The Long Baseline Neutrino Facility (LBNF) will deliver a 2.4 MW muon neutrino beam from Fermilab to the Deep Underground Neutrino Experiment (DUNE), requiring unprecedented precision in beamline alignment to achieve DUNE's neutrino oscillation measurement goals. Vertical misalignments of beamline components as small as 0.5 mm can contribute 6-7\% uncertainty in predicted neutrino flux, necessitating sub-0.1 mm alignment monitoring capabilities. The Horn Location Sensor (HLS) system employs frequency sweep interferometry (FSI) in a distributed hydrostatic leveling network to achieve the required precision under harsh radiation conditions up to 5000 kRad/year. Traditional FSI implementations suffer from laser sweep nonlinearities that degrade resolution and require computationally intensive post-processing corrections using gas reference cells. This work presents a real-time FPGA-based implementation of the HLS data acquisition and processing system using a sweep tracker interferometer for dynamic sweep linearization. The system utilizes a PYNQ-Z2 FPGA with programmable logic implementing parallel 16k-point FFT processing across four channels, synchronized by the sweep tracker signal to eliminate post-processing requirements. Spectral performance testing demonstrates significant improvements in peak sharpness compared to traditional fixed-frequency digitization. The FPGA implementation enables real-time displacement monitoring with processing speeds orders of magnitude faster than software-based approaches, essential for the operational requirements of LBNF's eventual distributed sensor network. This advancement in real-time FSI processing directly supports DUNE's precision neutrino physics program by providing the rapid feedback necessary for maintaining stringent beamline alignment tolerances during high-power beam operations.

Rossel, A. Jacob [Fermilab; Unlisted]↗

Design of the SLAC RCE Platform: A General Purpose ATCA Based Data Acquisition System

The SLAC RCE platform is a general purpose clustered data acquisition system implemented on a custom ATCA compliant blade, called the Cluster On Board (COB). The core of the system is the Reconfigurable Cluster Element (RCE), which is a system-on-chip design based upon the Xilinx Zynq family of FPGAs, mounted on custom COB daughter-boards. The Zynq architecture couples a dual core ARM Cortex A9 based processor with a high performance 28nm FPGA. The RCE has 12 external general purpose bi-directional high speed links, each supporting serial rates of up to 12Gbps. 8 RCE nodes are included on a COB, each with a 10Gbps connection to an on-board 24-port Ethernet switch integrated circuit. The COB is designed to be used with a standard full-mesh ATCA backplane allowing multiple RCE nodes to be tightly interconnected with minimal interconnect latency. Multiple shelves can be clustered using the front panel 10-gbps connections. The COB also supports local and inter-blade timing and trigger distribution. An experiment specific Rear Transition Module adapts the 96 high speed serial links to specific experiments and allows an experiment-specific timing and busy feedback connection. This coupling of processors with a high performance FPGA fabric in a low latency, multiple node cluster allows high speed data processing that can be easily adapted to any physics experiment. RTEMS and Linux are both ported to the module. The RCE has been used or is the baseline for several current and proposed experiments (LCLS, HPS, LSST, ATLAS-CSC, LBNE, DarkSide, ILC-SiD, etc).

43 PARTICLE ACCELERATORS↗

Modeling error PDF optimization based wavelet neural network modeling of dynamic system and its application in blast furnace ironmaking

In general, the modeling errors of dynamic system model are a set of random variables. The traditional performance index of modeling such as means square error (MSE) and root means square error (RMSE) can not fully express the connotation of modeling errors with stochastic characteristics both in the dimension of time domain and space domain. Therefore, the probability density function (PDF) is introduced to completely describe the modeling errors in both time scales and space scales. Based on it, a novel wavelet neural network (WNN) modeling method is proposed by minimizing the two-dimensional (2D) PDF shaping of modeling errors. First, the modeling error PDF by the tradional WNN is estimated using data-driven kernel density estimation (KDE) technique. Then, the quadratic sum of 2D deviation between the modeling error PDF and the target PDF is utilized as performance index to optimize the WNN model parameters by gradient descent method. Since the WNN has strong nonlinear approximation and adaptive capability, and all the parameters are well optimized by the proposed method, the developed WNN model can make the modeling error PDF track the target PDF, eventually. Simulation example and application in a blast furnace ironmaking process show that the proposed method has a higher modeling precision and better generalization ability compared with the conventional WNN modeling based on MSE criteria. Furthermore, the proposed method has more desirable estimation for modeling error PDF that approximates to a Gaussian distribution whose shape is high and narrow.

Modelling↗

Distributed Multi-GPU Community Detection on Exascale Computing Platforms

Community detection is a fundamental operation in graph mining, and by uncovering hidden structures and patterns within complex systems it helps solve fundamental problems pertaining to social networks, such as information diffusion, epidemics, and recommender systems. Scaling graph algorithms for massive networks becomes challenging on modern distributed-memory multi-GPU (Graphics Processing Unit) systems due to limitations such as irregular memory access patterns, load imbalances, higher communication-computation ratios, and cross-platform support. We present a novel algorithm HiPDPL-GPU (distributed parallel Louvain) to address these challenges. We conduct experiments involving different partitioning techniques to achieve optimized performance of HiPDPL-GPU on the two largest supercomputers: Frontier and Summit. Remarkably, HiPDPL-GPU processes a graph with 4.2 billion edges in less than 3 minutes using 1024 GPUs. Qualitatively performance of HiPDPL-GPU is similar or better compared to other state-of-the-art CPU- and GPU-based implementations. While prior GPU implementations have predominantly employed CUDA, our first-of-its-kind implementation for community detection is cross-platform, accommodating both AMD and NVIDIA GPUs.

graph algorithms, high performance comptuing↗