Search NASA⌕ Search

SEARCH · Search NASA

Results for “Parallelization Strategy”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22

Multigrid Reduction in Time for Nonlinear Parabolic Problems

The need for parallel-in-time is being driven by changes in computer architectures, where future speed-ups will be available through greater concurrency, but not faster clock speeds, which are stagnant.This leads to a bottleneck for sequential time marching schemes, because they lack parallelism in the time dimension. Multigrid Reduction in Time (MGRIT) is an iterative procedure that allows for temporal parallelism by utilizing multigrid reduction techniques and a multilevel hierarchy of coarse time grids. MGRIT has been shown to be effective for linear problems, with speedups of up to 50 times. The goal of this work is the efficient solution of nonlinear problems with MGRIT, where efficient is defined as achieving similar performance when compared to a corresponding linear problem. As our benchmark, we use the p-Laplacian, where p = 4 corresponds to a well-known nonlinear diffusion equation and p = 2 corresponds to our benchmark linear diffusion problem. When considering linear problems and implicit methods, the use of optimal spatial solvers such as spatial multigrid imply that the cost of one time step evaluation is fixed across temporal levels, which have a large variation in time step sizes. This is not the case for nonlinear problems, where the work required increases dramatically on coarser time grids, where relatively large time steps lead to worse conditioned nonlinear solves and increased nonlinear iteration counts per time step evaluation. This is the key difficulty explored by this paper. We show that by using a variety of strategies, most importantly, spatial coarsening and an alternate initial guess to the nonlinear time-step solver, we can reduce the work per time step evaluation over all temporal levels to a range similar with the corresponding linear problem. This allows for parallel scaling behavior comparable to the corresponding linear problem.

97 MATHEMATICS AND COMPUTING↗

Seamless Transition of Critical Infrastructures using Droop Controlled Grid-forming Inverters

Seamless recovery of power to critical infrastructures, after grid failure, is a crucial need arising in scenarios that are increasingly becoming more frequent. Here, this article proposes a seamless transition strategy using a single and unified mode-dependent droop controlled grid-forming inverters. The control strategy achieves the following objectives: 1) regulates the output active and reactive power by the droop-controlled inverters to a desired value while operating in on-grid mode; 2) seamless transition and recovery of power injections into the load after grid failure by inverters that operates in grid-forming mode all the time; 3) requires only a single bit of information on the grid/network status for the mode transition. A framework for assessing the stability of the system and to guide the choice of parameters for controllers is developed using control-oriented modeling. A controller hardware-in-the-loop-based real-time simulation study on a test system based on the realistic electrical network of a commercial-scale medical center is conducted for initial prototyping of the control strategy. A hardware experiment is conducted with two 3 - $\phi$, 480 -V, 125 -kVA grid-forming inverters, a 3 - $\phi$, 480 -V, 270 -kVA grid simulator, a physical grid switch, and a physical load bank. The experimental data establishes the effectiveness of the always grid-forming operation and control of inverters in meeting power delivery objectives when on-grid and off-grid under various kinds of loads and scenarios while minimizing transients during transitions. Furthermore, performance comparison with existing strategies showcases the advantage of the proposed strategy.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Ion-beam-induced bending of freestanding amorphous nanowires: The importance of the substrate material and charging

Ion-beam irradiation offers great flexibility and controllability in the construction of freestanding nanostructures with multiple advanced functionalities. Here, we present and discuss the bending of free-standing nanowires, against, towards, and ultimately parallel to a flux of directional ion irradiation. Bending components both along and perpendicular to the incident ion beam were observed, and the bending behavior was found to depend both on the ion beam scanning strategy and on the conductivity of the supporting substrate. This behavior is explained by an ion-irradiation-related electrostatic interaction. Our findings suggest the prospect of exploiting this technique to engineer 3D nanostructures for advanced applications.

36 MATERIALS SCIENCE↗

Energetic particle loss mechanisms in reactor-scale equilibria close to quasisymmetry

Abstract Collisionless physics primarily determines the transport of fusion-born alpha particles in 3D equilibria. Several transport mechanisms have been implicated in stellarator configurations, including stochastic diffusion due to class transitions, ripple trapping, and banana drift-convective orbits. Given the guiding center dynamics in a set of six quasihelical and quasiaxisymmetric equilibria, we perform a classification of trapping states and transport mechanisms. In addition to banana drift convection and ripple transport, we observe substantial non-conservation of the parallel adiabatic invariant which can cause losses through diffusive banana tip motion. Furthermore, many lost trajectories undergo transitions between trapping classes on longer time scales, either with periodic or irregular behavior. We discuss possible optimization strategies for each of the relevant transport mechanisms. We perform a comparison between fast ion losses and metrics for the prevalence of mechanisms such as banana-drift convection (Velasco et al 2021 Nucl. Fusion 61 116059), transitioning orbits, and wide orbit widths. Quasihelical configurations are found to have natural protection against ripple-trapping and diffusive banana tip motion leading to a reduction in prompt losses.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

MOOSE Enhancements Towards Delivery of an Integrated Framework

Started in 2008, the Mulitphysics Object Oriented Simulation Environment (MOOSE) project delivers a comprehensive framework for development of parallel, multiphysics simulation tools. MOOSE provides an object-oriented interface that allows scientists and engineers to quickly and efficiently describe all aspects of a physical system relevant to numerical modeling. The MOOSE software development strategy has been quite successful: over thirty different MOOSE-based applications are under development both internally and outside the DOE complex, and over forty different external institutions have licensed MOOSE. Several NEAMS products have been built using MOOSE, including BISON (Williamson et. al., 2012), MARMOT (Tonks et. al., 2012), RELAP-7 (Zou et. al., 2013) and RAVEN (Rabiti, 2013).

99 GENERAL AND MISCELLANEOUS↗

Gut microbiota and its therapeutic implications in tumor microenvironment interactions

The development of cancer is not just the growth and proliferation of a single transformed cell, but its tumor microenvironment (TME) also coevolves with it, which is primarily involved in tumor initiation, development, metastasis, and therapeutic responses. Recent years, TME has been emerged as a potential target for cancer diagnosis and treatment. However, the clinical efficacy of treatments targeting the TME, especially its specific components, remains insufficient. In parallel, the gut microbiome is an essential TME component that is crucial in cancer immunotherapy. Thus, assessing and constructing frameworks between the gut microbiota and the TME can significantly enhance the exploration of effective treatment strategies for various tumors. In this review the role of the gut microbiota in human cancers, including its function and relationship with various tumors was summarized. In addition, the interaction between the gut microbiota and the TME as well as its potential applications in cancer therapeutics was described. Furthermore, it was summarized that fecal microbiota transplantation, dietary adjustments, and synthetic biology to introduce gut microbiota-based medical technologies for cancer treatment. This review provides a comprehensive summary for uncovering the mechanism underlying the effects of the gut microbiota on the TME and lays a foundation for the development of personalized medicine in further studies.

Feng, Pengya↗

Virtual Cable Impedance based Load Sharing in a Microgrid for Parallel Connected Grid Forming Converters

This paper presents a novel approach to power sharing between direct connected grid-forming converters, utilizing virtual cable impedance and droop-based outer loop control. To enhance stability, resistive droop is implemented, while virtual cable impedance with non-zero resistive and inductive components ensures improved power sharing. The inner loop controller employs a Lyapunov energy function to achieve superior dynamic performance. The proposed control architecture is validated through comprehensive modeling and real-time processor-in-the-loop simulations, demonstrating its robustness and efficiency under various operating conditions. The results highlight the potential of this control strategy to improve the reliability and efficiency of renewable energy systems. Additionally, a comparative analysis with traditional methods underscores the advantages of the proposed approach in terms of stability and performance. The proposed control architecture offers a scalable and flexible solution for grid-forming converters, enabling seamless integration of renewable energy sources. Its robustness and adaptability make it an attractive solution for real-world applications. Furthermore, the approach can be extended to other power electronic systems, enhancing overall system performance and efficiency. By providing a reliable and efficient control strategy, this paper contributes to the advancement of renewable energy systems and their adoption in the energy sector. The proposed control strategy has far-reaching implications for the widespread adoption of renewable energy sources, enabling a more sustainable and efficient energy future. The overall system is modeled in MATLAB/Simulink and PLECS software domain.

grid forming converters (GFM)↗

Flexible User-Defined Domain Decomposition in Kilometer-Scale E3SM Land Model Simulation

The Energy Exascale Earth System Model (E3SM) Land Model (ELM) has been extended to kilometer-scale (km-ELM) resolutions, enabling high-fidelity simulations of terrestrial processes at 1 km x 1 km grid spacing. In ELM, domain decomposition partitions the computational domain across processors, ensuring efficient parallel execution. Currently, round-robin decomposition is applied, providing a straightforward way to distribute computational workload. As ELM continues evolving at the kilometer-scale (km-scale), particularly with integrating lateral flow modeling, decomposition strategies must also account for the increased workload and data movement. This paper introduces a flexible user-defined domain decomposition framework, allowing users to customize domain partitioning based on application requirements. The impact of different decomposition strategies is evaluated across various applications concerning computation, communication, and I/O. Results demonstrate that while 1D partitioning yields superior I/O performance, k-nearest neighbors (KNN) clustering effectively reduces inter-process communication overhead. This study lays the groundwork for scalable partitioning in large-scale land surface simulations, enhancing next-generation Earth system modeling.

Wang, Dali [ORNL] (ORCID:0000000168065108)↗

GPU-enabled extreme-scale turbulence simulations: Fourier pseudo-spectral algorithms at the exascale using OpenMP offloading

Fourier pseudo-spectral methods for nonlinear partial differential equations are of wide interest in many areas of advanced computational science, including direct numerical simulation of three-dimensional (3-D) turbulence governed by the Navier-Stokes equations in fluid dynamics. This paper presents a new capability for simulating turbulence at a new record resolution up to 35 trillion grid points, on the world's first exascale computer, Frontier, comprising AMD MI250x GPUs with HPE's Slingshot interconnect and operated by the US Department of Energy's Oak Ridge Leadership Computing Facility (OLCF). Key programming strategies designed to take maximum advantage of the machine architecture involve performing almost all computations on the GPU which has the same memory capacity as the CPU, performing all-to-all communication among sets of parallel processes directly on the GPU, and targeting GPUs efficiently using OpenMP offloading for intensive number-crunching including 1-D Fast Fourier Transforms (FFT) performed using AMD ROCm library calls. With 99% of computing power on Frontier being on the GPU, leaving the CPU idle leads to a net performance gain via avoiding the overhead of data movement between host and device except when needed for some I/O purposes. Memory footprint including the size of communication buffers for MPI_ALLTOALL is managed carefully to maximize the largest problem size possible for a given node count. Detailed performance data including separate contributions from different categories of operations to the elapsed wall time per step are reported for five grid resolutions, from 2048 3 on a single node to 32768 3 on 4096 or 8192 nodes out of 9408 on the system. Both 1D and 2D domain decompositions which divide a 3D periodic domain into slabs and pencils respectively are implemented. The present code suite (labeled by the acronym GESTS, GPUs for Extreme Scale Turbulence Simulations) achieves a figure of merit (in grid points per second) exceeding goals set in the Center for Accelerated Application Readiness (CAAR) program for Frontier. The performance attained is highly favorable in both weak scaling and strong scaling, with notable departures only for 2048 3 where communication is entirely intra-node, and for 32768 3 , where a challenge due to small message sizes does arise. Communication performance is addressed further using a lightweight test code that performs all-to-all communication in a manner matching the full turbulence simulation code. Performance at large problem sizes is affected by both small message size due to high node counts as well as dragonfly network topology features on the machine, but is consistent with official expectations of sustained performance on Frontier. Overall, although not perfect, the scalability achieved at the extreme problem size of 32768 3 (and up to 8192 nodes — which corresponds to hardware rated at just under 1 exaflop/sec of theoretical peak computational performance) is arguably better than the scalability observed using prior state-of-the-art algorithms on Frontier's predecessor machine (Summit) at OLCF. New science results for the study of intermittency in turbulence enabled by this code and its extensions are to be reported separately in the near future.

3D fast Fourier transform↗

An adaptive scalable fully implicit algorithm based on stabilized finite element for reduced visco-resistive MHD

The magnetohydrodynamics (MHD) equations are continuum models used in the study of a wide range of plasma physics systems, including the evolution of complex plasma dynamics in tokamak disruptions. However, efficient numerical solution methods for MHD are extremely challenging due to disparate time and length scales, strong hyperbolic phenomena, and nonlinearity. Additionally, therefore the development of scalable, implicit MHD algorithms and high-resolution adaptive mesh refinement strategies is of considerable importance. In this work, we develop a high-order stabilized finite-element algorithm for the reduced visco-resistive MHD equations based on the MFEM finite element library (mfem.org). The scheme is fully implicit, solved with the Jacobian-free Newton-Krylov (JFNK) method with a physics-based preconditioning strategy. Our preconditioning strategy is a generalization of the physics-based preconditioning methods in Chacón et al. (2002) to adaptive, stabilized finite elements. Algebraic multigrid methods are used to invert sub-block operators to achieve scalability. A parallel adaptive mesh refinement scheme with dynamic load-balancing is implemented to efficiently resolve the multi-scale spatial features of the system. Our implementation uses the MFEM framework, which provides arbitrary-order polynomials and flexible adaptive conforming and non-conforming meshes capabilities. Results demonstrate the accuracy, efficiency, and scalability of the implicit scheme in the presence of large scale disparity. The potential of the AMR approach is demonstrated on an island coalescence problem in the high Lundquist-number regime (≥ 10 7 ) with the successful resolution of plasmoid instabilities and thin current sheets.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Multiscale modeling of polyisoprene on graphite

The local dynamics and the conformational properties of polyisoprene next to a smooth graphite surface constructed by graphene layers are studied by a multiscale methodology. First, fully atomistic molecular dynamics simulations of oligomers next to the surface are performed. Subsequently, Monte Carlo simulations of a systematically derived coarse-grained model generate numerous uncorrelated structures for polymer systems. A new reverse backmapping strategy is presented that reintroduces atomistic detail. Finally, multiple extensive fully atomistic simulations with large systems of long macromolecules are employed to examine local dynamics in proximity to graphite. Polyisoprene repeat units arrange close to a parallel configuration with chains exhibiting a distribution of contact lengths. Efficient Monte Carlo algorithms with the coarse-grain model are capable of sampling these distributions for any molecular weight in quantitative agreement with predictions from atomistic models. Furthermore, molecular dynamics simulations with well-equilibrated systems at all length-scales support an increased dynamic heterogeneity that is emerging from both intermolecular interactions with the flat surface and intramolecular cooperativity. This study provides a detailed comprehensive picture of polyisoprene on a flat surface and consists of an effort to characterize such systems in atomistic detail.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A Versatile Simulated Data Transport Layer for in Situ Workflows Performance Evaluation

In situ processing does not only allow scientific applications to face the explosion in data volume and velocity but also to address the time constraints of many simulation-analysis workflows by providing scientists with early insights about their applications at runtime. Multiple frameworks implement the concept of a data transport layer (DTL) to enable such in situ workflows. These tools are very versatile, directly or indirectly access the data generated on the same node, another node of the same compute cluster, or a completely distinct node, and allow data publishers and subscribers to run on the same computing resources or not. This versatility puts on researchers the onus of taking key decisions related to resource allocation and how to transport data to ensure the most efficient execution of their in situ workflows. However, domain scientists and workflow practitioners lack the appropriate tools to assess the respective performance of particular design and deployment options. In this paper we introduce a versatile simulated DTL designed to provide researchers with insights on the respective performance of different execution scenarios of in situ workflows. This open-source, standalone library builds on the SimGrid toolkit and can be linked to any SimGrid-based simulator. It facilitates the evaluation of the performance behavior, at scale, of different data transport configurations and the study of the effects of resource allocation strategies. We demonstrate the scalability, versatility, and accuracy of this simulated DTL by reproducing the execution of two synthetic benchmarks and of a real-world in situ workflow composed of an MPI application and a parallel data analysis. Results of simulations run on a single core show that the proposed library can simulate the interactions of tens of thousands of simulated processes deployed on two interconnected commodity clusters in a few seconds, and the execution by a thousand simulated processes of an in situ workflow in less than three minutes.

Suter, Fred [ORNL] (ORCID:0000000319021955)↗

SU-E-T-395: Multi-GPU-Based VMAT Treatment Plan Optimization Using a Column-Generation Approach

Purpose: GPU has been employed to speed up VMAT optimizations from hours to minutes. However, its limited memory capacity makes it difficult to handle cases with a huge dose-deposition-coefficient (DDC) matrix, e.g. those with a large target size, multiple arcs, small beam angle intervals and/or small beamlet size. We propose multi-GPU-based VMAT optimization to solve this memory issue to make GPU-based VMAT more practical for clinical use. Methods: Our column-generation-based method generates apertures sequentially by iteratively searching for an optimal feasible aperture (referred as pricing problem, PP) and optimizing aperture intensities (referred as master problem, MP). The PP requires access to the large DDC matrix, which is implemented on a multi-GPU system. Each GPU stores a DDC sub-matrix corresponding to one fraction of beam angles and is only responsible for calculation related to those angles. Broadcast and parallel reduction schemes are adopted for inter-GPU data transfer. MP is a relatively small-scale problem and is implemented on one GPU. One headand- neck cancer case was used for test. Three different strategies for VMAT optimization on single GPU were also implemented for comparison: (S1) truncating DDC matrix to ignore its small value entries for optimization; (S2) transferring DDC matrix part by part to GPU during optimizations whenever needed; (S3) moving DDC matrix related calculation onto CPU. Results: Our multi-GPU-based implementation reaches a good plan within 1 minute. Although S1 was 10 seconds faster than our method, the obtained plan quality is worse. Both S2 and S3 handle the full DDC matrix and hence yield the same plan as in our method. However, the computation time is longer, namely 4 minutes and 30 minutes, respectively. Conclusion: Our multi-GPU-based VMAT optimization can effectively solve the limited memory issue with good plan quality and high efficiency, making GPUbased ultra-fast VMAT planning practical for real clinical use.

60 APPLIED LIFE SCIENCES↗

Lifting and Dropping VMs to Dynamically Transition Between Time- and Space-sharing for Large-Scale HPC Systems

As HPC environments increasingly integrate with edge based systems, system architectures will need to handle a broader class of workloads and scheduling requirements. One result of this shift will be the need to simultaneously support bulk-synchronous parallel (BSP) and on-demand service based applications on the same infrastructure. This in turn will require that future resource management approaches utilize both space-shared as well as time-shared resource scheduling strategies. In this work we introduce the concept of "VM-lifting'' (and its inverse "VM-Dropping'') which allows dynamically switching an HPC workload between space-shared and time-shared scheduling regimes. Our work targets co-kernel based HPC system software environments, in which multiple specialized OS kernels execute natively on dedicated physical resource partitions inside a single compute node. With VM-lifting, a native co-kernel can be migrated at runtime to and from locally hosted Virtual Machine Environments due to changing scheduling requirements of the node. This allows an HPC node to be dynamically (re-)configured as either a time-shared Infrastructure-as-a-Service (IaaS) resource or a dedicated space shared resource based on the current workload demands. We have implemented this approach in the context of the Hobbes Exascale System Software stack and have demonstrated that a node can be reconfigured with minimal impact on the running applications.

Gordon, Nick↗

Fabrication of Scaffold-Based 3D Magnetic Nanowires for Domain Wall Applications

Three-dimensional magnetic nanostructures hold great potential to revolutionize information technologies and to enable the study of novel physical phenomena. In this work, we describe a hybrid nanofabrication process combining bottom-up 3D nano-printing and top-down thin film deposition, which leads to the fabrication of complex magnetic nanostructures suitable for the study of new 3D magnetic effects. First, a non-magnetic 3D scaffold is nano-printed using Focused Electron Beam Induced Deposition; then a thin film magnetic material is thermally evaporated onto the scaffold, leading to a functional 3D magnetic nanostructure. Scaffold geometries are extended beyond recently developed single-segment geometries by introducing a dual-pitch patterning strategy. Additionally, by tilting the substrate during growth, low-angle segments can be patterned, circumventing a major limitation of this nano-printing process; this is demonstrated by the fabrication of ‘staircase’ nanostructures with segments parallel to the substrate. The suitability of nano-printed scaffolds to support thermally evaporated thin films is discussed, outlining the importance of including supporting pillars to prevent deformation during the evaporation process. Employing this set of methods, a set of nanostructures tailored to precisely match a dark-field magneto-optical magnetometer have been fabricated and characterized. This work demonstrates the versatility of this hybrid technique and the interesting magnetic properties of the nanostructures produced, opening a promising route for the development of new 3D devices for applications and fundamental studies.

36 MATERIALS SCIENCE↗

Modeling of streamflow in a 30 km long reach spanning 5 years using OpenFOAM 5.x

Abstract. Developing accurate and efficient modeling techniques for streamflow at the tens-of-kilometers spatial scale and multi-year temporal scale is critical for evaluating and predicting the impact of climate- and human-induced discharge variations on river hydrodynamics. However, achieving such a goal is challenging because of limited surveys of streambed hydraulic roughness, uncertain boundary condition specifications, and high computational costs. We demonstrate that accurate and efficient three-dimensional (3-D) hydrodynamic modeling of natural rivers at 30 km and 5-year scales is feasible using the following three techniques within OpenFOAM, an open-source computational fluid dynamics platform: (1) generating a distributed hydraulic roughness field for the streambed by integrating water-stage observation data, a rough wall theory, and a local roughness optimization and adjustment strategy; (2) prescribing the boundary condition for the inflow and outflow by integrating precomputed results of a one-dimensional (1-D) hydraulic model with the 3-D model; and (3) reducing computational time using multiple parallel runs constrained by 1-D inflow and outflow boundary conditions. Streamflow modeling for a 30 km long reach in the Columbia River (CR) over 58 months can be achieved in less than 6 d using 1.1 million CPU hours. The mean error between the modeled and the observed water stages for our simulated CR reach ranges from −16 to 9 cm (equivalent to approximately ±7 % relative to the average water depth) at seven locations during most of the years between 2011 and 2019. We can reproduce the velocity distribution measured by the acoustic Doppler current profiler (ADCP). The correlation coefficients of the depth-averaged velocity between the model and ADCP measurements are in the range between 0.71 and 0.83 at 75 % of the survey cross sections. With the validated model, we further show that the relative importance of dynamic pressure versus hydrostatic pressure varies with discharge variations and topography heterogeneity. Given the model's high accuracy and computational efficiency, the model framework provides a generic approach to evaluate and predict the impacts of climate- and human-induced discharge variations on river hydrodynamics at tens-of-kilometers and decadal scales.

58 GEOSCIENCES↗

Tusas: A fully implicit parallel approach for coupled phase-field equations

In this study, we develop a fully-coupled, fully-implicit approach for phase-field modeling of solidification in metals and alloys. Predictive simulation of solidification in pure metals and metal alloys remains a significant challenge in the field of materials science, as microstructure formation during the solidification process plays a critical role in the properties and performance of the solid material. Our simulation approach consists of a finite element spatial discretization of the fully-coupled nonlinear system of partial differential equations at the microscale, which is treated implicitly in time with a preconditioned Jacobian-free Newton-Krylov method. The approach is algorithmically scalable as well as efficient due to an effective preconditioning strategy based on algebraic multigrid and block factorization. We implement this approach in the open-source Tusas framework, which is a general, flexible tool developed in C++ for solving coupled systems of nonlinear partial differential equations. The performance of our approach is analyzed in terms of algorithmic scalability and efficiency, while the computational performance of Tusas is presented in terms of parallel scalability and efficiency on emerging heterogeneous architectures. We demonstrate that modern algorithms, discretizations, and computational science, and heterogeneous hardware provide a robust route for predictive phase-field simulation of microstructure evolution during additive manufacturing.

97 MATHEMATICS AND COMPUTING↗

FLEET: Flexible Efficient Ensemble Training for Heterogeneous Deep Neural Networks

Parallel training of an ensemble of Deep Neural Networks (DNN) on a cluster of nodes is an effective approach to shorten the process of neural network architecture search and hyper-parameter tuning for a given learning task. Prior efforts have shown that data sharing, where the common preprocessing operation is shared across the DNN training pipelines, saves computational resources and improves pipeline efficiency. Data sharing strategy, however, performs poorly for a heterogeneous set of DNNs where each DNN has varying computational needs and thus different training rate and convergence speed. This paper proposes FLEET, a flexible ensemble DNN training framework for efficiently training a heterogeneous set of DNNs. We build FLEET via several technical innovations. We theoretically prove that an optimal resource allocation is NP-hard and propose a greedy algorithm to efficiently allocate resources for training each DNN with data sharing. We integrate data-parallel DNN training into ensemble training to mitigate the differences in training rates and introduce checkpointing into this context to address the issue of different convergence speeds. Experiments show that FLEET significantly improves the training efficiency of DNN ensembles without compromising the quality of the result.

Guan, Hui↗