Search NASA⌕ Search

SEARCH · Search NASA

Results for “Parallel Optimization Algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Nonlinear multiobjective and dynamic real-time predictive optimization for optimal operation of baseload power plants under variable renewable energy

Considering the increase of disruptive variable renewable energy penetration into the power grid, this article focuses on the investigation of a multiobjective and dynamic real-time optimization framework to address the cycling of large-scale power plants under renewable penetration. In this framework, a parallelized particle swarm optimization step is first performed to generate feasible initial points. Then, a multiobjective and dynamic real-time optimization formulation generates optimal trajectories. Further, the benefit of predictive capability is investigated for the dynamic component, which introduces the novel nonlinear multiobjective and dynamic real-time predictive optimization approach. Two multiobjective formulations to obtain Pareto front optimal in real time are explored: the modified Tchebycheff-based weighted metric and ϵ-constraint methods. Economic and environmental objectives are considered in this study. A novel topical discussion on the intersection of dynamic real-time optimization with model predictive control is also presented. The developed framework is successfully applied to a baseload coal-fired power plant with postcombustion CO 2 capture. Results indicate that the approach can be deployed for a large-scale system if automatic differentiation, model reduction, and parallelization are adopted to improve computational tractability, with computational improvement up to 120-folds after performing these steps. Finally, market and carbon policies showed an impact on the optimal compromise between the objectives with an additional 63 ton of CO 2 captured under favorable market conditions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

A massively parallel and scalable multi-CPU material point method

Harnessing the power of modern multi-GPU architectures, we present a massively parallel simulation system based on the Material Point Method (MPM) for simulating physical behaviors of materials undergoing complex topological changes, self-collision, and large deformations. Our system makes three critical contributions. First, we introduce a new particle data structure that promotes coalesced memory access patterns on the GPU and eliminates the need for complex atomic operations on the memory hierarchy when writing particle data to the grid. Second, we propose a kernel fusion approach using a new Grid-to-Particles-to-Grid (G2P2G) scheme, which efficiently reduces GPU kernel launches, improves latency, and significantly reduces the amount of global memory needed to store particle data. Finally, we introduce optimized algorithmic designs that allow for efficient sparse grids in a shared memory context, enabling us to best utilize modern multi-GPU computational platforms for hybrid Lagrangian-Eulerian computational patterns. We demonstrate the effectiveness of our method with extensive benchmarks, evaluations, and dynamic simulations with elastoplasticity, granular media, and fluid dynamics. In comparisons against an open-source and heavily optimized CPU-based MPM codebase [Fang et al. 2019] on an elastic sphere colliding scene with particle counts ranging from 5 to 40 million, our GPU MPM achieves over 100x per-time-step speedup on a workstation with an Intel 8086K CPU and a single Quadro P6000 GPU, exposing exciting possibilities for future MPM simulations in computer graphics and computational science. Moreover, compared to the state-of-the-art GPU MPM method [Hu et al. 2019a], we not only achieve 2x acceleration on a single GPU but our kernel fusion strategy and Array-of-Structs-of-Array (AoSoA) data structure design also generalizes to multi-GPU systems. Our multi-GPU MPM exhibits near-perfect weak and strong scaling with 4 GPUs, enabling performant and large-scale simulations on a 10243 grid with close to 100 million particles with less than 4 minutes per frame on a single 4-GPU workstation and 134 million particles with less than 1 minute per frame on an 8-GPU workstation.

Wang, Xinlei↗
DOE OSTI2022

EQC: Ensembled Quantum Computing for Variational Quantum Algorithms

Variational quantum algorithms (VQA), which are comprised of a classical optimizer and a parameterized quantum circuit, emerges as one of the most promising approaches of harvesting quantum power in the noisy-intermediate-scale-quantum (NISQ) era. However, the deployment of VQAs on today's NISQ devices often faces considerable system noise and prohibitively slow training speeds. On the other hand, the expensive supporting sources and infrastructure make quantum computers extremely keen on high utilization. In this paper, we propose a novel way of thinking about a quantum backend: rather than relying on one physical device which tends to introduce platform-specific noise and bias, a quantum ensemble, which distributes quantum tasks across parallel devices, can serve as a virtualized quantum computer for offering reduced noise levels through an adaptive mixture and also provide significantly improved training speeds through parallelization. With this idea, we build a distributive VQA optimization framework called DVQA, serving as the first effort in adopting parallel quantum devices for cooperative VQA training. To further constraint noise and speed-up convergence, we design a model for individual NISQ devices concerning their properties and running conditions, and propose a weighting mechanism for regularizing the returned gradients. Extensive evaluations on 10 IBM-Q quantum devices using the VQE example show that the distributive VQA training framework can substantially boost the training speed by 10.5x on average (up to 86x and at least 5.2x) with improved training accuracy.

Stein, Samuel A.↗
DOE OSTI2024

OPER: Optimality-Guided Embedding Table Parallelization for Large-scale Recommendation Model

With the sharp increasing volume of user data, Deep Learning Recommendation Model (DLRM) becomes an indispensable infrastructure in large technology companies. However, large-scale DLRM on the multi-GPU platform is still inefficient due to unbalanced workload partitioning and intensive inter-GPU communication. To this end, we propose OPER, an OPtimality guided Embedding table placement for large-scale Recommendation model training and inference. OPER explores the potential of mitigating remote memory access latency in DLRM through fine-grained embedding table placement. Specifically, OPER proposes a theoretical modeling that builds up the relationship between EMT placement and the embedding communication latency in both training and inference. OPER proves the NP hardness of finding the optimal embedding table placement and proposes a heuristic algorithm that yields near optimal placement. OPER implements a SHMEM-based embedding table training system and a unified embedding index mapping to support fine-grained embedding table sharding and placement. Comprehensive experiments reveal that OPER achieves on average 3.4× and 5.1× speedup on training and inference respectively over state-of-the-art DLRM frameworks.

Wang, Zheng↗
DOE OSTI2024

Parallel interior-point solver for block-structured nonlinear programs on SIMD/GPU architectures

Here, we investigate how to port the standard interior-point method to new exascale architectures for block-structured nonlinear programs with state equations. Computationally, we decompose the interior-point algorithm into two successive operations: the evaluation of the derivatives and the solution of the associated Karush-Kuhn-Tucker (KKT) linear system. Our method accelerates both operations using two levels of parallelism. First, we distribute the computations on multiple processes using coarse parallelism. Second, each process uses SIMD/GPU accelerators locally to accelerate the operations using fine-grained parallelism. The KKT system is reduced by eliminating the inequalities and the state variables from the corresponding equations. We demonstrate our method's capability on the supercomputer Polaris, a testbed for the future exascale Aurora system. Each node is equipped with four GPUs, a setup amenable to our two-level approach. Our experiments on the stochastic optimal power flow problem show that the reduction method is 50x faster than the sparse linear solver HSL MA57 running in serial on the CPU, and 6x faster than Pardiso running in parallel on CPU on the same number of processes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Optimization of Thermal Conductance at Interfaces Using Machine Learning Algorithms

We report optimization of thermal transport across the interface of two different materials is critical to micro-/nanoscale electronic, photonic, and phononic devices. Although several examples of compositional intermixing at the interfaces having a positive effect on interfacial thermal conductance (ITC) have been reported, an optimum arrangement has not yet been determined because of the large number of potential atomic configurations and the significant computational cost of evaluation. On the other hand, computation-driven materials design efforts are rising in popularity and importance. Yet, the scalability and transferability of machine learning models remain as challenges in creating a complete pipeline for the simulation and analysis of large molecular systems. In this work we present a scalable Bayesian optimization framework, which leverages dynamic spawning of jobs through the Message Passing Interface (MPI) to run multiple parallel molecular dynamics simulations within a parent MPI job to optimize heat transfer at the silicon and aluminum (Si/Al) interface. We found a maximum of 50% increase in the ITC when introducing a two-layer intermixed region that consists of a higher percentage of Si. Because of the random nature of the intermixing, the magnitude of increase in the ITC varies. We observed that both homogeneity/heterogeneity of the intermixing and the intrinsic stochastic nature of molecular dynamics simulations account for the variance in ITC.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Scalable branching on dual decomposition of stochastic mixed-integer programming problems

In this work, we present a scalable branching method for the dual decomposition of stochastic mixed-integer programming. Our new branching method is based on the branching method proposed by Caroe and Schultz that creates branching disjunctions on first-stage variables only. We propose improvements to the process for creating branching disjunctions, including (1) branching on the optimal solutions of the Dantzig-Wolfe reformulation of the restricted master problem and (2) using a more comprehensive (yet simple) measure for the dispersions associated with subproblem solution infeasibility. We prove that the proposed branching process leads to an algorithm that terminates finitely, and we provide conditions under which globally optimal solutions can be identified after termination. We have implemented our new branching method, as well as the Caroe-Schultz method and a branch-and-price method, in the open-source software package DSP. Using SIPLIB test instances, we present extensive numerical results to demonstrate that the proposed branching method significantly reduces the number of node subproblems and solution times.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Exploiting Geometric Partitioning in Task Mapping for Parallel Computes

We present a new method for mapping applications' MPI tasks to cores of a parallel computer such that applications' communication time is reduced. We address the case of sparse node allocation, where the nodes assigned to a job are not necessarily located in a contiguous block nor within close proximity to each other in the network, although our methods generalize to contiguous allocations as well. The goal is to assign tasks to cores so that interdependent tasks are performed by "nearby' cores, thus lowering the distance messages must travel, the amount of congestion in the network, and the overall cost of communication. Our new method applies a geometric partitioning algorithm to both the tasks and the processors, and assigns task parts to the corresponding processor parts. We also present a number of algorithmic optimizations that exploit specific features of the network or application. We show that, for the structured finite difference mini-application MiniGhost, our mapping methods reduced communication time up to 75% relative to MiniGhost's default mapping on 128K cores of a Cray XK7 with sparse allocation. For the atmospheric modeling code E3SM/HOMME, our methods reduced communication time up to 31% on 32K cores of an IBM BlueGene/Q with contiguous allocation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

MPMC and MCMD: Free High-Performance Simulation Software for Atomistic Systems

Advancements in parallel computing and hardware have allowed computational exploration of chemical systems of interest with unprecendented accuracy and efficiency. The typical development of molecular simulation software is initially inspired by a particular scientific inquiry. The softwares presented herein, MPMC (Massively Parallel Monte Carlo) and MCMD (Monte Carlo/Molecular Dynamics) were born out of a pursuit to simulate condensed phase physical and chemical interactions in porous materials. MPMC first began in 2005 and has been used for dozens of published results in the literature but has not yet been introduced in a standalone paper. MCMD is a more recent expansion and re-write with some published work, focused on adding molecular dynamics algorithms for transport and other time-dependent properties of chemical systems. Each software functions as a standalone with some unique and other overlapping features. Here, a driving aim of this work is to consider periodic, long-range polarization effects in classical simulation, and both codes are optimized to perform such calculations. Sample results will be presented which highlight methodically that inclusion of explicit polarization produces simulation results with predictive power which in general are in greater agreement with experiment than non-polarizable analogous simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Multichannel Analysis of Surface Waves Accelerated (MASWAccelerated): Software for efficient surface wave inversion using MPI and GPUs

Multichannel Analysis of Surface Waves (MASW) is a technique frequently used in geotechnical engineering and engineering geophysics to infer 1D layered models of seismic shear wave velocities in the top tens to hundreds of meters of the subsurface. We aim to accelerate MASW calculations by capitalizing on modern computer hardware available in the workstations of most engineers: multiple cores and graphics processing units (GPUs). We propose new parallel and GPU accelerated algorithms for computing 1D MASW inversion, and provide software implementations in C using Message Passing Interface (MPI) and CUDA. These algorithms take advantage of sparsity that arises in the problem, and the work balance between processes considers typical data trends. We compare our methods to an existing open source Matlab MASW tool. Our serial C implementation achieves a 2x speedup over the Matlab software, and we continue to see improvements by parallelizing the problem with MPI. Here we see nearly perfect strong and weak scaling for uniform data, and improve strong scaling for realistic data by repartitioning the problem to process mapping. By utilizing GPUs available on most modern workstations, we observe an additional 1.3x speedup over the serial C implementation on the first use of the method. We typically repeatedly evaluate theoretical dispersion curves as part of an optimization procedure, and on the GPU the kernel can be cached for faster reuse on later runs. We observe a 3.2x speedup on the cached GPU runs compared to the serial C runs. This work is the first open-source parallel or GPU-accelerated software tool for MASW imaging, and should enable geotechnical engineers to fully utilize all computer hardware at their disposal.

58 GEOSCIENCES↗
DOE OSTI2019

The Continuous Readout Stream of the MicroBooNE Liquid Argon Time Projection Chamber for Detection of Supernova Neutrinos

The MicroBooNE Continuous Readout Stream is a parallel readout of the MicroBooNE liquid argon time projection chamber (LArTPC) which enables detection of non-beam events such as those from a supernova neutrino burst. The low energies of these neutrinos and the intense cosmic-ray flux due to the near-surface detector location makes triggering on these events very challenging. Instead, MicroBooNE has pioneered a novel approach for detecting supernova neutrinos relying on a delayed trigger generated by SNEWS (the Supernova Early Warning System). The continuous readout of the LArTPC is achieved through the use of real-time compression algorithms (zero suppression and Huffman compression) implemented in an FPGA (field-programmable gate array) in the readout electronics. We present the results of the optimization of the data reduction algorithms, and their operational performance. In order to demonstrate the capability of the continuous stream to detect low-energy electrons, a sample of Michel electrons from stopping cosmic-ray muons is reconstructed and compared to a similar sample from the lossless triggered readout stream.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

The continuous readout stream of the MicroBooNE liquid argon time projection chamber for detection of supernova burst neutrinos

The MicroBooNE continuous readout stream is a parallel readout of the MicroBooNE liquid argon time projection chamber (LArTPC) which enables detection of non-beam events such as those from a supernova neutrino burst. The low energies of the supernova neutrinos and the intense cosmic-ray background flux due to the near-surface detector location makes triggering on these events very challenging. Instead, MicroBooNE relies on a delayed trigger generated by SNEWS (the Supernova Early Warning System) for detecting supernova neutrinos. The continuous readout of the LArTPC generates large data volumes, and requires the use of real-time compression algorithms (zero suppression and Huffman compression) implemented in an FPGA (field-programmable gate array) in the readout electronics. In this paper we present the results of the optimization of the data reduction algorithms, and their operational performance. To demonstrate the capability of the continuous stream to detect low-energy electrons, a sample of Michel electrons from stopping cosmic-ray muons is reconstructed and compared to a similar sample from the lossless triggered readout stream.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Porting Classical Approaches for Quantum Simulations to Quantum Computers

Simulating quantum many-body systems is one of the most promising problems in which we might anticipate that quantum computers should show quantum advantage. Unfortunately, there is still a gap between this promise and actual practice. New quantum algorithms need to be developed and the current quantum algorithms have various difficulties - e.g efficient state preparation - which must be overcome and improved upon. In many cases, classical approaches need to be ported over to quantum devices. In this project we have developed a suite of new quantum algorithms which makes progress in this regard. We developed a new optimization scheme for variational quantum eigensolvers, UBOS, which mitigates problems with local minimas and barren plateaus while improving convergence to the ground state by an order of magnitude. We developed a new way to utilize qubitization to find ground states of nearly frustration-free Hamiltonians faster than all previous methods. We developed a series of state preparation techniques which helps initialize parameterized quantum circuits into reasonable starting points on which quantum algorithms are then applied. In addition to the development of novel algorithms, it is critical to have classical simulation techniques for approximately simulating quantum circuits which can be used to benchmark and understand quantum algorithms. Toward that end, we developed a novel POVM formalism to simulate quantum circuits as well as exemplify the massive parallelization of tensor network methodologies. Finally, we developed physical understanding of entanglement phase transitions such as many-body localization and random tensor networks.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2018

Scaling Deep Learning workloads: NVIDIA DGX-1/Pascal and Intel Knights Landing

Deep Learning (DL) algorithms have become ubiquitous in data analytics. As a result, major computing vendors—including NVIDIA, Intel, AMD, and IBM—have architectural road maps influenced by DL workloads. Furthermore, several vendors have recently advertised new computing products as accelerating large DL workloads. Unfortunately, it is difficult for data scientists to quantify the potential of these different products. Here, this article provides a performance and power analysis of important DL workloads on two major parallel architectures: NVIDIA DGX-1 (eight Pascal P100 GPUs interconnected with NVLink) and Intel Knights Landing (KNL) CPUs interconnected with Intel Omni-Path or Cray Aries. Our evaluation consists of a cross section of convolutional neural net workloads: CifarNet, AlexNet, GoogLeNet, and ResNet50 topologies using the Cifar10 and ImageNet datasets. The workloads are vendor-optimized for each architecture. We use sequentially equivalent implementations to maintain iso-accuracy between parallel and sequential DL models. Our analysis indicates that although GPUs provide the highest overall performance, the gap can close for some convolutional networks; and the KNL can be competitive in performance/watt. We find that NVLink facilitates scaling efficiency on GPUs. However, its importance is heavily dependent on neural network architecture. Furthermore, for weak-scaling—sometimes encouraged by restricted GPU memory—NVLink is less important.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Concurrent Relaxation through Accelerated Deep Learning

CRADL captures performance metrics of machine learning algorithms operating on mesh data from multiphysics codes This proxy application is a tool to explore scalability of inference on HPC platforms, and also gather performance metrics for inference on new machine learning specific hardware. CRADL is designed to give users as fine a control as possible over an inference simulation. Users may select the number of cycles, amount of data, and batch size to pass to the accelerator of choice. Additionally the user may select a number of performance optimization libraries and flags. CRADL comes packaged with a repository of anonymized multi-physics simulation data, as well as a pretrained model for inference. The code allows a user to load their own pre-trained model and data if they wish. The code can operate in multiple parallelization schemes, with performance enhancing options such as half-precision libraries, PyTorch benchmarking, and pinned memory with non-blocking data transfers.

Zieb, KristoferJ.↗
DOE OSTI2022

A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining

Transformers are considered one of the most important deep learning models since 2018, in part because it establishes state-of-the-art (SOTA) records and could potentially replace existing Deep Neural Networks (DNNs). Despite the remarkable triumphs, the prolonged turnaround time of Transformer models is a widely recognized roadblock. The variety of sequence lengths imposes additional computing overhead where inputs need to be zero-padded to the maximum sentence length in the batch to accommodate the parallel computing platforms. This paper targets the field-programmable gate array (FPGA) and proposes a coherent sequence length adaptive algorithm–hardware co-design for Transformer acceleration. Particularly, we develop a hardware-friendly sparse attention operator and a length-aware hardware resource scheduling algorithm. The proposed sparse attention operator brings the complexity of attention-based models down to linear complexity and alleviates the off-chip memory traffic. The proposed length-aware resource hardware scheduling algorithm dynamically allocates the hardware resources to fill up the pipeline slots and eliminates bubbles for NLP tasks. Experiments show that our design has very small accuracy loss and has 80.2 × and 2.6 × speedup compared to CPU and GPU implementation, and 4 × higher energy efficiency than state-of-the-art GPU accelerator optimized via CUBLAS GEMM.

Peng, Hongwu↗
DOE OSTI2025

Direct numerical simulations for hybrid rocket boundary layers: Performance modeling and scaling

This paper presents a comprehensive performance and scaling analysis of direct numerical simulations for reacting boundary layers, focusing on slab burner configurations. Using a PETSc-based finite volume CFD framework, the study evaluates the scalability and computational cost of flow, chemistry, and radiation evaluations across 2D and 3D simulations. Polymethyl methacrylate (PMMA) is the fuel with pure O 2 as the oxidizer, modeled using a detailed chemical kinetics mechanism with 113 species and 660 reactions. A ray-tracing-based radiation solver, designed for distributed memory applications, is implemented to model radiation heat transfer. Parallel scalability is analyzed for the coupled flow, chemistry, and radiation heat transfer processes. Weak and strong scaling studies are conducted on up to 15,000 computational ranks, revealing robust performance when flow cells exceed 200 per rank. Chemistry evaluations dominate the computational cost in large 3D simulations, accounting for approximately 40% of the total runtime, while flow processes contribute around 35%, and radiation solver contributions remain below 10% due to reduced evaluation frequencies. GPU accelerated chemistry evaluation, implemented with Zero-RK, demonstrates significant promise, achieving up to a 4x speedup for workloads exceeding 30,000 cells per GPU. However, diminishing returns are observed for smaller workloads due to CPU-GPU communication overhead. This study identifies key challenges, including memory bottlenecks and the effects of domain partitioning on flow scalability, while highlighting the potential of GPU-accelerated chemistry to reduce computational costs. In conclusion, these findings provide realizable run configurations for 2D, 3D, and GPU-accelerated cases, offering insights for optimizing reactive flow solvers.

CFD Scalability↗
DOE OSTI2019

Monte Carlo Second- and Third-Order Many-Body Green’s Function Methods with Frequency-Dependent, Nondiagonal Self-Energy

We fully develop the Monte Carlo many-body Green’s function (MC-GF) method with the following enhancements: (1) The truncation order of the perturbation expansion of the Dyson self-energy is raised from the second order (MC-GF2) to the third order (MC-GF3) with the aid of a computerized procedure to enumerate and transform all 84 third-order Goldstone diagrams into Monte Carlo integrable expressions and then into central processing unit (CPU)/graphical processing unit (GPU)-parallel computer codes. (2) An efficient algorithm is proposed that computes all off-diagonal and diagonal elements of the MC-GF2 and MC-GF3 self-energy matrices by common subexpression elimination. (3) The frequency-independent approximation is lifted by introducing a method that computes frequency derivatives of the MC-GF2 and MC-GF3 self-energies up to any arbitrarily high order at nearly no additional computational cost. (4) The imaginary-time integration in the Laplace-transformed expressions of the self-energy is carried out stochastically (instead of using a quadrature in the previous implementations), resulting in a 50- to 200-fold speedup. (5) The efficiency of the redundant-walker convergence acceleration scheme is analyzed numerically, and the guidelines are established to select an optimal number of walkers for maximal efficiency. When such an optimal number is used, the cost per sample is constant of molecular size on either many CPUs or many GPUs. (6) The computational cost to obtain a binding energy within a given statistical uncertainty is observed to increase as (tentatively) O(n 4 ) and O(n 5 ) of molecular size (n is the number of electrons) with and without the diagonal approximation, respectively, as opposed to O(n 5 ) and O(n 6 ) of the corresponding deterministic algorithms. With this method applied to the electron binding energies of C 60 , we show that the third-order corrections to the self-energies are much greater in electron binding energies than in ground-state energies. As a result, they display a sign of oscillatory convergence toward experimental results, not necessarily improving the agreement with increasing perturbation order, justifying MC-GF3 and motivating even higher-order methods.

Algorithms↗