Search NASA⌕ Search

SEARCH · Search NASA

Results for “parallel computer architecture”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,081 records · Page 60

Machine Learning Algorithm Performance on the Lucata Computer

A new parallel computing paradigm (processor in memory, or PIM) has recently become available, one that uses many lightweight threads, and where each thread migrates automatically to the memory used by that thread. Our effort focuses on understanding how suitable this architecture is for our application, and whether the hardware can sustain speedups as high as the system size permits. In particular we explore the kind of code optimizations needed, and how well optimized code scales. This paper describes some of the those optimizations, and the payoff in terms of scaling.

Kogge, Peter↗

Parallelization of Lower-Upper Symmetric Gauss-Seidel Method for Chemically Reacting Flow

Development of technologies for exploration of the solar system has revived an interest in computational simulation of chemically reacting flows since planetary probe vehicles exhibit non-equilibrium phenomena during the atmospheric entry of a planet or a moon as well as the reentry to the Earth. Stability in combustion is essential for new propulsion systems. Numerical solution of real-gas flows often increases computational work by an order-of-magnitude compared to perfect gas flow partly because of the increased complexity of equations to solve. Recently, as part of Project Columbia, NASA has integrated a cluster of interconnected SGI Altix systems to provide a ten-fold increase in current supercomputing capacity that includes an SGI Origin system. Both the new and existing machines are based on cache coherent non-uniform memory access architecture. Lower-Upper Symmetric Gauss-Seidel (LU-SGS) relaxation method has been implemented into both perfect and real gas flow codes including Real-Gas Aerodynamic Simulator (RGAS). However, the vectorized RGAS code runs inefficiently on cache-based shared-memory machines such as SGI system. Parallelization of a Gauss-Seidel method is nontrivial due to its sequential nature. The LU-SGS method has been vectorized on an oblique plane in INS3D-LU code that has been one of the base codes for NAS Parallel benchmarks. The oblique plane has been called a hyperplane by computer scientists. It is straightforward to parallelize a Gauss-Seidel method by partitioning the hyperplanes once they are formed. Another way of parallelization is to schedule processors like a pipeline using software. Both hyperplane and pipeline methods have been implemented using openMP directives. The present paper reports the performance of the parallelized RGAS code on SGI Origin and Altix systems.

Yoon, Seokkwan↗

TORCH Computational Reference Kernels - A Testbed for Computer Science Research

For decades, computer scientists have sought guidance on how to evolve architectures, languages, and programming models in order to improve application performance, efficiency, and productivity. Unfortunately, without overarching advice about future directions in these areas, individual guidance is inferred from the existing software/hardware ecosystem, and each discipline often conducts their research independently assuming all other technologies remain fixed. In today's rapidly evolving world of on-chip parallelism, isolated and iterative improvements to performance may miss superior solutions in the same way gradient descent optimization techniques may get stuck in local minima. To combat this, we present TORCH: A Testbed for Optimization ResearCH. These computational reference kernels define the core problems of interest in scientific computing without mandating a specific language, algorithm, programming model, or implementation. To compliment the kernel (problem) definitions, we provide a set of algorithmically-expressed verification tests that can be used to verify a hardware/software co-designed solution produces an acceptable answer. Finally, to provide some illumination as to how researchers have implemented solutions to these problems in the past, we provide a set of reference implementations in C and MATLAB.

97 MATHEMATICS AND COMPUTING↗

Characterizing the performance of node-aware strategies for irregular point-to-point communication on heterogeneous architectures

Supercomputer architectures are trending toward higher computational throughput due to the inclusion of heterogeneous compute nodes. These multi-GPU nodes increase on-node computational efficiency, while also increasing the amount of data to be communicated and the number of potential data flow paths. In this work, we characterize the performance of irregular point-to-point communication with MPI on heterogeneous compute environments through performance modeling, demonstrating the limitations of standard communication strategies for both device-aware and staging-through-host communication techniques. Presented models suggest staging communicated data through host processes then using node-aware communication strategies for high inter-node message counts. Notably, the models also predict that node-aware communication utilizing all available CPU cores to communicate inter-node data leads to the most performant strategy when communicating with a high number of nodes. Furthermore, model validation is provided via a case study of irregular point-to-point communication patterns in distributed sparse matrix–vector products. Importantly, we include a discussion on the implications model predictions have on communication strategy design for emerging supercomputer architectures.

97 MATHEMATICS AND COMPUTING↗

A PARALLEL MONTE CARLO CODE FOR SIMULATING COLLISIONAL N-BODY SYSTEMS

We present a new parallel code for computing the dynamical evolution of collisional N-body systems with up to N {approx} 10{sup 7} particles. Our code is based on the Henon Monte Carlo method for solving the Fokker-Planck equation, and makes assumptions of spherical symmetry and dynamical equilibrium. The principal algorithmic developments involve optimizing data structures and the introduction of a parallel random number generation scheme as well as a parallel sorting algorithm required to find nearest neighbors for interactions and to compute the gravitational potential. The new algorithms we introduce along with our choice of decomposition scheme minimize communication costs and ensure optimal distribution of data and workload among the processing units. Our implementation uses the Message Passing Interface library for communication, which makes it portable to many different supercomputing architectures. We validate the code by calculating the evolution of clusters with initial Plummer distribution functions up to core collapse with the number of stars, N, spanning three orders of magnitude from 10{sup 5} to 10{sup 7}. We find that our results are in good agreement with self-similar core-collapse solutions, and the core-collapse times generally agree with expectations from the literature. Also, we observe good total energy conservation, within {approx}< 0.04% throughout all simulations. We analyze the performance of the code, and demonstrate near-linear scaling of the runtime with the number of processors up to 64 processors for N = 10{sup 5}, 128 for N = 10{sup 6} and 256 for N = 10{sup 7}. The runtime reaches saturation with the addition of processors beyond these limits, which is a characteristic of the parallel sorting algorithm. The resulting maximum speedups we achieve are approximately 60 Multiplication-Sign , 100 Multiplication-Sign , and 220 Multiplication-Sign , respectively.

79 ASTRONOMY AND ASTROPHYSICS↗

FLEET: Flexible Efficient Ensemble Training for Heterogeneous Deep Neural Networks

Parallel training of an ensemble of Deep Neural Networks (DNN) on a cluster of nodes is an effective approach to shorten the process of neural network architecture search and hyper-parameter tuning for a given learning task. Prior efforts have shown that data sharing, where the common preprocessing operation is shared across the DNN training pipelines, saves computational resources and improves pipeline efficiency. Data sharing strategy, however, performs poorly for a heterogeneous set of DNNs where each DNN has varying computational needs and thus different training rate and convergence speed. This paper proposes FLEET, a flexible ensemble DNN training framework for efficiently training a heterogeneous set of DNNs. We build FLEET via several technical innovations. We theoretically prove that an optimal resource allocation is NP-hard and propose a greedy algorithm to efficiently allocate resources for training each DNN with data sharing. We integrate data-parallel DNN training into ensemble training to mitigate the differences in training rates and introduce checkpointing into this context to address the issue of different convergence speeds. Experiments show that FLEET significantly improves the training efficiency of DNN ensembles without compromising the quality of the result.

Guan, Hui↗

DSIM: A distributed simulator

Discrete event-driven simulation makes it possible to model a computer system in detail. However, such simulation models can require a significant time to execute. This is especially true when modeling large parallel or distributed systems containing many processors and a complex communication network. One solution is to distribute the simulation over several processors. If enough parallelism is achieved, large simulation models can be efficiently executed. This study proposes a distributed simulator called DSIM which can run on various architectures. A simulated test environment is used to verify and characterize the performance of DSIM. The results of the experiments indicate that speedup is application-dependent and, in DSIM's case, is also dependent on how the simulation model is distributed among the processors. Furthermore, the experiments reveal that the communication overhead of ethernet-based distributed systems makes it difficult to achieve reasonable speedup unless the simulation model is computation bound.

Goswami, Kumar K.↗

Collaborative Research: A Model of Partially Ionized Plasma Flows with Kinetic Treatment of Neutral Atoms and Nonthermal Ions

Interactions of flows of partially ionized, magnetized plasma are frequently accompanied by the presence of both thermal and non-thermal (pickup) ion components. Such interactions cannot be modeled using traditional MHD equations and require more advanced approaches to treat them. If a nonthermal component of ions is formed due to charge exchange and collisions between the thermal (core) ions and neutrals, it experiences the action of magnetic field, its distribution function is isotropized, and it soon acquires the velocity of the ambient plasma without being thermodynamically equilibrated. This situation, e. g., takes place in the outer heliosphere –- the part of interstellar space beyond the solar system whose properties are determined by the solar wind interaction with the local interstellar medium. This is also possible in laboratory, at million degrees and above, when plasma is conducting electricity far too well, which makes Ohmic heating ineffective. To attain the target temperatures one needs additional heating eventually playing a dominant role. Among such sources is a so-called neutral particle beam heating. This is a wide-spread technique (Joint European Torus and International Thermonuclear Experimental Reactor experiments) based on the injection of powerful beams of neutral atoms into ohmically preheated plasma. In this project we have investigated the energy and density separation between the thermal and nonthermal components in the solar wind and interstellar plasmas. A new model has been developed in which we solve the ideal MHD equations for mixture of all ions and the kinetic Boltzmann equation to describe the transport of neutral atoms. As a separate capability, we can treat the flow of neutral atoms in a multi-component fashion, where neutral atoms born in each thermodynamically distinct region are governed by the Euler gas dynamic equations. We also describe the behavior of pickup ions either kinetically, using the Fokker--Planck equation, or as a separate fluid. Our numerical simulations have demonstrated that pickup ions play a major role in the interaction of the solar wind and (partially ionized) interstellar medium plasmas. Our teams have investigated the stability of the surface (the heliopause) that separates the solar wind from the local interstellar medium, the transport of galactic cosmic rays, the properties of the heliotail flow, and modifications to the bow wave in front of the heliopause due to charge exchange between the neutral H atoms born in the solar wind and interstellar ions. Modeling results have been validated against observational data, such as obtained by the Interstellar Boundary Explorer (IBEX), and made it possible to shed light on the structure of energetic neutral atom maps created by this spacecraft.. We have also demonstrated that charge-exchange modulated heliosphere is a source of anisotropy of the multi-TeV cosmic ray flux observed in a number of Earth-bound air shower experiments. Newly developed codes are implemented within a Multi-Scale Fluid-Kinetic Simulation Suite (MS-FLUKSS), a publicly available code being developed by our team for over 12 years. MS-FLUKSS scales well up to 160,000 computing cores and has been ported on major supercomputers in the country. Efficient parallelization and data choreography in the continuum simulation modules are provided by Chombo, an adaptive mesh refinement framework managed by Phillip Colella's team at LBNL. We have implemented in-house, hybrid (MPI+OpenMP) parallelization of the kinetic modules that solve the Boltzmann equation with a Monte Carlo method. Currently, the kinetic modules are being rewritten to take advantage of the modern CPU-GPU supercomputer architecture. The scope of the project allowed us to enhance plasma research and education in such broad, multidisciplinary field as physics of partially ionized plasma and its application to space physics and fusion science. Besides the impact on the modeling of complex physical systems, our approach to computational resource management for complex codes utilizing multiple algorithm technologies appears to be a major advance on current approaches. The development of sophisticated resource management will be essential for all future modeling efforts that incorporate a diversity of scales and physical processes. Our effort provided leadership in promoting computational science and plasma physics within the UAH and FIT campuses and, through the training of a broad spectrum of scientists and engineers, fostering new technologies across the country.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

The Navier-Stokes computer

The Navier-Stokes computer (NSC) has been developed for solving problems in fluid mechanics involving complex flow simulations that require more speed and capacity than provided by current and proposed Class VI supercomputers. The machine is a parallel processing supercomputer with several new architectural elements which can be programmed to address a wide range of problems meeting the following criteria: (1) the problem is numerically intensive, and (2) the code makes use of long vectors. A simulation of two-dimensional nonsteady viscous flows is presented to illustrate the architecture, programming, and some of the capabilities of the NSC.

Nosenchuck, D. M.↗

Fault-tolerant computer architecture based on INMOS transputer processor

Redundant processing was used for several years in mission flight systems. In these systems, more than one processor performs the same task at the same time but only one processor is actually in real use. A fault-tolerance computer architecture based on the features provided by INMOS Transputers is presented. The Transputer architecture provides several communication links that allow data and command communication with other Transputers without the use of a bus. Additionally the Transputer allows the use of parallel processing to increase the system speed considerably. The processor architecture consists of three processors working in parallel keeping all the processors at the same operational level but only one processor is in real control of the process. The design allows each Transputer to perform a test to the other two Transputers and report the operating condition of the neighboring processors. A graphic display was developed to facilitate the identification of any problem by the user.

Ortiz, Jorge L.↗

Research in Parallel Algorithms and Software for Computational Aerosciences

Phase 1 is complete for the development of a computational fluid dynamics CFD) parallel code with automatic grid generation and adaptation for the Euler analysis of flow over complex geometries. SPLITFLOW, an unstructured Cartesian grid code developed at Lockheed Martin Tactical Aircraft Systems, has been modified for a distributed memory/massively parallel computing environment. The parallel code is operational on an SGI network, Cray J90 and C90 vector machines, SGI Power Challenge, and Cray T3D and IBM SP2 massively parallel machines. Parallel Virtual Machine (PVM) is the message passing protocol for portability to various architectures. A domain decomposition technique was developed which enforces dynamic load balancing to improve solution speed and memory requirements. A host/node algorithm distributes the tasks. The solver parallelizes very well, and scales with the number of processors. Partially parallelized and non-parallelized tasks consume most of the wall clock time in a very fine grain environment. Timing comparisons on a Cray C90 demonstrate that Parallel SPLITFLOW runs 2.4 times faster on 8 processors than its non-parallel counterpart autotasked over 8 processors.

Domel, Neal D.↗

Research in Parallel Algorithms and Software for Computational Aerosciences

Phase I is complete for the development of a Computational Fluid Dynamics parallel code with automatic grid generation and adaptation for the Euler analysis of flow over complex geometries. SPLITFLOW, an unstructured Cartesian grid code developed at Lockheed Martin Tactical Aircraft Systems, has been modified for a distributed memory/massively parallel computing environment. The parallel code is operational on an SGI network, Cray J90 and C90 vector machines, SGI Power Challenge, and Cray T3D and IBM SP2 massively parallel machines. Parallel Virtual Machine (PVM) is the message passing protocol for portability to various architectures. A domain decomposition technique was developed which enforces dynamic load balancing to improve solution speed and memory requirements. A host/node algorithm distributes the tasks. The solver parallelizes very well, and scales with the number of processors. Partially parallelized and non-parallelized tasks consume most of the wall clock time in a very fine grain environment. Timing comparisons on a Cray C90 demonstrate that Parallel SPLITFLOW runs 2.4 times faster on 8 processors than its non-parallel counterpart autotasked over 8 processors.

Domel, Neal D.↗

WARP3D-Release 10.8: Dynamic Nonlinear Analysis of Solids using a Preconditioned Conjugate Gradient Software Architecture

This report describes theoretical background material and commands necessary to use the WARP3D finite element code. WARP3D is under continuing development as a research code for the solution of very large-scale, 3-D solid models subjected to static and dynamic loads. Specific features in the code oriented toward the investigation of ductile fracture in metals include a robust finite strain formulation, a general J-integral computation facility (with inertia, face loading), an element extinction facility to model crack growth, nonlinear material models including viscoplastic effects, and the Gurson-Tver-gaard dilatant plasticity model for void growth. The nonlinear, dynamic equilibrium equations are solved using an incremental-iterative, implicit formulation with full Newton iterations to eliminate residual nodal forces. The history integration of the nonlinear equations of motion is accomplished with Newmarks Beta method. A central feature of WARP3D involves the use of a linear-preconditioned conjugate gradient (LPCG) solver implemented in an element-by-element format to replace a conventional direct linear equation solver. This software architecture dramatically reduces both the memory requirements and CPU time for very large, nonlinear solid models since formation of the assembled (dynamic) stiffness matrix is avoided. Analyses thus exhibit the numerical stability for large time (load) steps provided by the implicit formulation coupled with the low memory requirements characteristic of an explicit code. In addition to the much lower memory requirements of the LPCG solver, the CPU time required for solution of the linear equations during each Newton iteration is generally one-half or less of the CPU time required for a traditional direct solver. All other computational aspects of the code (element stiffnesses, element strains, stress updating, element internal forces) are implemented in the element-by- element, blocked architecture. This greatly improves vectorization of the code on uni-processor hardware and enables straightforward parallel-vector processing of element blocks on multi-processor hardware.

Koppenhoefer, Kyle C.↗

Computer Electromagnetics and Supercomputer Architecture

The dramatic increase in performance over the last decade for microporcessor computations is compared with that for the supercomputer computations. This performance, the projected performance, and a number of other issues such as cost and the inherent pysical limitations in curent supercomputer technology have naturally led to parallel supercomputers and ensemble of interconnected microprocessors.

Electromagnetics Microprocessors Computers↗

A Review of Lightweight Thread Approaches for High Performance Computing

High-level, directive-based solutions are becoming the programming models (PMs) of the multi/many-core architectures. Several solutions relying on operating system (OS) threads perfectly work with a moderate number of cores. However, exascale systems will spawn hundreds of thousands of threads in order to exploit their massive parallel architectures and thus conventional OS threads are too heavy for that purpose. Several lightweight thread (LWT) libraries have recently appeared offering lighter mechanisms to tackle massive concurrency. In order to examine the suitability of LWTs in high-level runtimes, we develop a set of microbenchmarks consisting of commonlyfound patterns in current parallel codes. Moreover, we study the semantics offered by some LWT libraries in order to expose the similarities between different LWT application programming interfaces. This study reveals that a reduced set of LWT functions can be sufficient to cover the common parallel code patterns and that those LWT libraries perform better than OS threads-based solutions in cases where task and nested parallelism are becoming more popular with new architectures.

Castello, Adrian↗

Collaborative Research: A Model of Partially Ionized Plasma Flows with Kinetic Treatment of Neutral Atoms and Nonthermal Ions

Interactions of flows of partially ionized, magnetized plasma are frequently accompanied by the presence of both thermal and non-thermal (pickup) ion components. Such interactions cannot be modeled using traditional MHD equations and require more advanced approaches to treat them. If a nonthermal component of ions is formed due to charge exchange and collisions between the thermal (core) ions and neutrals, it experiences the action of magnetic field, its distribution function is isotropized, and it soon acquires the velocity of the ambient plasma without being thermodynamically equilibrated. This situation, e. g., takes place in the outer heliosphere - the part of interstellar space beyond the solar system whose properties are determined by the solar wind interaction with the local interstellar medium. This is also possible in laboratory, at million degrees and above, when plasma is conducting electricity far too well, which makes Ohmic heating ineffective. To attain the target temperatures one needs additional heating eventually playing a dominant role. Among such sources is a so-called neutral particle beam heating. This is a wide-spread technique (Joint European Torus and International Thermonuclear Experimental Reactor experiments) based on the injection of powerful beams of neutral atoms into ohmically preheated plasma. In this project we have investigated the energy and density separation between the thermal and nonthermal components in the solar wind and interstellar plasmas. A new model has been developed in which we solve the ideal MHD equations for mixture of all ions and the kinetic Boltzmann equation to describe the transport of neutral atoms. As a separate capability, we can treat the flow of neutral atoms in a multi-component fashion, where neutral atoms born in each thermodynamically distinct regions are governed by the Euler gas dynamic equations. We also describe the behavior of pickup ions either kinetically, using the Fokker–Planck equation, or as a separate fluid. Our numerical simulations have demonstrated that pickup ions play a major role in the interaction of the solar wind and (partially ionized) interstellar medium plasmas. Our teams have investigated the stability of the surface (the heliopause) that separates the solar wind from the local interstellar medium, the transport of galactic cosmic rays, the properties of the heliotail flow, and modifications to the bow wave in front of the heliopause due to charge exchange between the neutral H atoms born in the solar wind and interstellar ions. Modeling results have been validated against observational data, such as obtained by the Interstellar Boundary Explorer (IBEX), and made it possible to shed light on the structure of energetic neutral atom maps created by this spacecraft.. We have also demonstrated that charge-exchange modulated heliosphere is a source of anisotropy of the multi-TeV cosmic ray flux observed in a number of Earth-bound air shower experiments. Newly developed codes are implemented within a Multi-Scale Fluid-Kinetic Simulation Suite (MS-FLUKSS), a publicly available code being developed by our team for over 12 years. MS-FLUKSS scales well up to 160,000 computing cores and has been ported on major supercomputers in the country. Efficient parallelization and data choreography in the continuum simulation modules are provided by Chombo, an adaptive mesh refinement framework managed by Phillip Colella’s team at LBNL. We have implemented in-house, hybrid (MPI+OpenMP) parallelization of the kinetic modules that solve the Boltzmann equation with a Monte Carlo method. Currently, the kinetic modules are being rewritten to take advantage of the modern CPU-GPU supercomputer architecture. The scope of the project allowed us to enhance plasma research and education in such broad, multidis- ciplinary field as physics of partially ionized plasma and its application to space physics and fusion science. Besides the impact on the modeling of complex physical systems, our approach to computational resource management for complex codes utilizing multiple algorithm technologies appears to be a major advance on current approaches. The development of sophisticated resource management will be essential for all future modeling efforts that incorporate a diversity of scales and physical processes. Our effort provided leadership in promoting computational science and plasma physics within the UAH and FIT campuses and, through the training of a broad spectrum of scientists and engineers, foster new technologies across the country.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Parallelized Kalman-Filter-Based Reconstruction of Particle Tracks on Many-Core Architectures

Faced with physical and energy density limitations on clock speed, contemporary microprocessor designers have increasingly turned to on-chip parallelism for performance gains. Algorithms should accordingly be designed with ample amounts of fine-grained parallelism if they are to realize the full performance of the hardware. This requirement can be challenging for algorithms that are naturally expressed as a sequence of small-matrix operations, such as the Kalman filter methods widely in use in high-energy physics experiments. In the High-Luminosity Large Hadron Collider (HL-LHC), for example, one of the dominant computational problems is expected to be finding and fitting charged-particle tracks during event reconstruction; today, the most common track-finding methods are those based on the Kalman filter. Experience at the LHC, both in the trigger and offline, has shown that these methods are robust and provide high physics performance. Previously we reported the significant parallel speedups that resulted from our efforts to adapt Kalman-filter-based tracking to many-core architectures such as Intel Xeon Phi. Here we report on how effectively those techniques can be applied to more realistic detector configurations and event complexity.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

GSoFa: Scalable Sparse Symbolic LU Factorization on GPUs

Decomposing a matrix $\mathbf {A}$ into a lower matrix $\mathbf {L}$ and an upper matrix $\mathbf {U}$, which is also known as LU decomposition, is an essential operation in numerical linear algebra. For a sparse matrix, LU decomposition often introduces more nonzero entries in the $\mathbf {L}$ and $\mathbf {U}$ factors than in the original matrix. A symbolic factorization step is needed to identify the nonzero structures of $\mathbf {L}$ and $\mathbf {U}$ matrices. Attracted by the enormous potentials of the Graphics Processing Units (GPUs), an array of efforts have surged to deploy various LU factorization steps except for the symbolic factorization, to the best of our knowledge, on GPUs. This article introduces gSoFa, the first GPU-based symbolic factorization design with the following three optimizations to enable scalable LU symbolic factorization for nonsymmetric pattern sparse matrices on GPUs. First, here we introduce a novel fine-grained parallel symbolic factorization algorithm that is well suited for the Single Instruction Multiple Thread (SIMT) architecture of GPUs. Second, we tailor supernode detection into a SIMT friendly process and strive to balance the workload, minimize the communication and saturate the GPU computing resources during supernode detection. Third, we introduce a three-pronged optimization to reduce the excessive space consumption problem faced by multi-source concurrent symbolic factorization. Taken together, gSoFa achieves up to 31× speedup from 1 to 44 Summit nodes (6 to 264 GPUs) and outperforms the state-of-the-art CPU project, on average, by 5×. Notably, gSoFa also achieves up to 47 percent of the peak memory throughput of a V100 GPU in the Summit Supercomputer.

97 MATHEMATICS AND COMPUTING↗