SEARCH · Search NASA

Results for “Memory Bandwidth”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

All sources NASA NTRS DOE OSTI

DOE OSTI2019

YGM

YGM provides a simplified distributed memory communication framework based on a mailbox abstraction for HPC applications. YGM is designed for irregular data-driven applications that communicate small message payloads in an all-to-all fashion. YGM aggregates and routes messages into larger payloads to gain greater bandwidth utilization on modern HPC interconnects. YGM has been demonstrated up to 131k cores of IBM BG/P Sequoia and 96k cores of Catalyst at LLNL, and contains a set of graph-inspired benchmarks.

Priest, BenjaminW.↗

DOE OSTI2019

Manycore Parallel Computing for a Hybridizable Discontinuous Galerkin Nested Multigrid Method

In this work, we present a parallel computing strategy for a hybridizable discontinuous Galerkin (HDG) nested geometric multigrid (GMG) solver. Parallel GMG solvers require a combination of coarse-grain and fine-grain parallelism to improve time-to-solution performance. In this work we focus on fine-grain parallelism. We use Intel's second generation Xeon Phi (Knights Landing) manycore processor. The GMG method achieves ideal convergence rates of 0.2 or less, for high polynomial orders. A matrix free (assembly free) technique is exploited to save considerable memory usage and increase arithmetic intensity. HDG enables static condensation, and due to the discontinuous nature of the discretization, we developed a matrix vector multiply routine that does not require any costly synchronizations or barriers. Our algorithm is able to attain 80% of peak bandwidth performance for higher order polynomials. This is possible due to the data locality inherent in the HDG method. Very high performance is realized for high order schemes, due to good arithmetic intensity, which declines as the order is reduced.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

In‐memory integration of existing software components for parallel adaptive unstructured mesh workflows

Summary Reliable mesh‐based simulations are needed to solve complex engineering problems. Mesh adaptivity can increase reliability by reducing discretization errors but requires multiple software components to exchange information. Often, components exchange information by reading and writing a common file format. This file‐based approach becomes a problem on massively parallel computers where filesystem bandwidth is a critical performance bottleneck. Our approach using data streams and component interfaces avoids the filesystem bottleneck. In this paper, we present these techniques and their use for coupling mesh adaptivity to the PHASTA computational fluid dynamics solver, the Albany multi‐physics framework, and the Omega3P linear accelerator frequency analysis applications. Performance results are reported on up to 16,384 cores of an Intel Knights Landing‐based system.

Smith, Cameron W.↗

DOE OSTI2016

A new ATLAS muon CSC readout system with system on chip technology on ATCA platform

The ATLAS muon Cathode Strip Chamber (CSC) backend readout system has been upgraded during the LHC 2013-2015 shutdown to be able to handle the higher Level-1 trigger rate of 100 kHz and the higher occupancy at Run-2 luminosity. The readout design is based on the Reconfigurable Cluster Element (RCE) concept for high bandwidth generic DAQ implemented on the Advanced Telecommunication Computing Architecture (ATCA) platform. The RCE design is based on the new System on Chip XILINX ZYNQ series with a processor-centric architecture with ARM processor embedded in FPGA fabric and high speed I/O resources. Together with auxiliary memories, all of these components form a versatile DAQ building block that can host applications tapping into both software and firmware resources. The Cluster on Board (COB) ATCA carrier hosts RCE mezzanines and an embedded Fulcrum network switch to form an online DAQ processing cluster. More compact firmware solutions on the ZYNQ for high speed input and output fiberoptic links and TTC allowed the full system of 320 input links from the 32 chambers to be processed by 6 COBs in one ATCA shelf. The full system was installed in September 2014. In conclusion, we will present the RCE/COB design concept, the firmware and software processing architecture, and the experience from the intense commissioning for LHC Run 2.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2018

The Design, Deployment, and Evaluation of the CORAL Pre-Exascale Systems

CORAL, the Collaboration of Oak Ridge, Argonne and Livermore, is fielding two similar IBM systems, Summit and Sierra, with NVIDIA GPUs that will replace the existing Titan and Sequoia systems. Summit and Sierra are currently ranked No. 1 and No. 3, respectively on the Top500 list. We discuss the design and key differences of the systems. Our evaluation of the systems highlights the following. Applications that fit in HBM see the most benefit and may prefer more GPUs; however, for some applications, the CPU-GPU bandwidth is more important than the number of GPUs. The node-local burst buffer scales linearly, and can achieve a 4X improvement over the parallel file system for large jobs; smaller jobs, however, may benefit from writing directly to the PFS. Finally, several CPU, network and memory bound analytics and GPU-bound deep learning codes achieve up to a 11X and 79X speedup/node, respectively over Titan.

Vazhkudai, Sudharshan↗

DOE OSTI2017

GPU-Centric Communication on NVIDIA GPU Clusters with InfiniBand: A Case Study with OpenSHMEM

GPUs have become an essential component for building compute clusters with high compute density and high performance per watt. As such clusters scale to have 1000s of GPUs, efficiently moving data between the GPUs becomes imperative to get maximum performance. NVSHMEM is an implementation of the OpenSHMEM standard for NVIDIA GPU clusters which allows communication to be issued from inside GPU kernels. In earlier work, we have shown how NVSHMEM can be used to achieve better application performance on GPUs connected through PCIe or NVLink. As part of this effort, we implement IB verbs for Mellanox InfiniBand adapters in CUDA. We evaluate different design alternatives, taking into consideration the relaxed memory model, automatic memory access coalescing and thread hierarchy on the GPU. We also consider correctness issues that arise in these designs. We take advantage of these designs transparently or through API extensions in NVSHMEM. With micro-benchmarks, we show that a Nvidia Pascal P100 GPU is able saturate the network bandwidth using only one or two of its 56 available streaming multiprocessors (SM). On a single GPU using a single IB EDR adapter, we achieve a throughput of around 90 million messages per second. In addition, we implement a 2dstencil application kernel using NVSHMEM and compare its performance with a CUDA-aware MPI-based implementation that uses GPUDirect RDMA. Speedups in the range of 23% to 42% are seen for input sizes large enough to fill the occupancy of Nvidia Pascal P100 GPUs on 2 to 4 nodes indicating that there are gains to be had by eliminating the CPU from the communication path when all computation runs on the GPU.

Potluri, Sreeram↗

DOE OSTI2020

Checkpoint triggering in a computer system

According to an aspect, a method for triggering creation of a checkpoint in a computer system includes executing a task in a processing node of the computer system. A monitoring block size is determined for the checkpoint. A checkpoint interval is determined based on the monitoring block size, a checkpoint bandwidth, and a failure rate of the computer system. Based on determining that the checkpoint interval has elapsed, the checkpoint including state data of the task is created to enable restarting execution of the task upon a restart operation. The state data of the checkpoint is restored from a memory responsive to detecting an error condition at the processing node. Execution of the task is restarted in the processing node based on the state data restored from the memory.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Refactoring the elastic–viscous–plastic solver from the sea ice model CICE v6.5.1 for improved performance

This study focuses on the performance of the elastic–viscous–plastic (EVP) dynamical solver within the sea ice model, CICE v6.5.1. The study has been conducted in two steps. First, the standard EVP solver was extracted from CICE for experiments with refactored versions, which are used for performance testing. Second, one refactored version was integrated and tested in the full CICE model to demonstrate that the new algorithms do not significantly impact the physical results. The study reveals two dominant bottlenecks, namely (1) the number of Message Parsing Interface (MPI) and Open Multi-Processing (OpenMP) synchronization points required for halo exchanges during each time step combined with the irregular domain of active sea ice points and (2) the lack of single-instruction, multiple-data (SIMD) code generation. The standard EVP solver has been refactored based on two generic patterns. The first pattern exposes how general finite differences on masked multi-dimensional arrays can be expressed in order to produce significantly better code generation by changing the memory access pattern from random access to direct access. The second pattern takes an alternative approach to handle static grid properties. The measured single-core performance improvement is more than a factor of 5 compared to the standard implementation. The refactored implementation of strong scales on the Intel® Xeon® Scalable Processors series node until the available bandwidth of the node is used. For the Intel® Xeon® CPU Max series, there is sufficient bandwidth to allow the strong scaling to continue for all the cores on the node, resulting in a single-node improvement factor of 35 over the standard implementation. This study also demonstrates improved performance on GPU processors.

58 GEOSCIENCES↗

DOE OSTI2019

Waterfall counters and an application to architectural vulnerability factor estimation

Described herein are waterfall counters and an application to architectural vulnerability factor (AVF) estimation. Waterfall counters count events that are generated at event generation logic. The waterfall counters are a combination of small, fast counters local to the event generation logic, and larger, global counters in fast memory. The local counters can be saturation or oscillation counters. When a local counter is saturated or evicted, the value from the local counter is added to the global counter. This addition can be done using logic local to the local or global counter. The waterfall counters provide a full-accuracy event count without the high bandwidth that is needed to maintain the global counters. An AVF estimation can be determined based on ratios from counts of read events, write events, and total events using the waterfall counters.

Gupta, Manish↗

DOE OSTI2018

Evaluating and Optimizing OpenCL Base64 Data Unpacking Kernel with FPGA

Development of applications using OpenCL targeting FPGAs is an emerging approach on heterogeneous computing systems. This paper uses the data unpacking algorithm in Base64 encoding as a case study to present programming and optimization techniques, and experimental results of the OpenCL-based implementations on an FPGA. We explain the algorithm and evaluate the performance of the kernel implementations with Intel's FPGA OpenCL SDK. The experimental results show kernel vectorization and duplication are two optimization techniques that can improve the kernel performance. The performance of kernel duplication is also closely related to the local work size. Our experiment shows 16-lane vectorization increases the bandwidth by a factor of 2 to 10 for large input data sizes. Moreover, the performance of kernel duplication using 16 compute units is 40% to 1.5% less than that of kernel vectorization depending on the input size. Tuning the local work size can improve the kernel performance by a factor of 3 to 23. For this kernel, using local memory is not an effective technique to improve the kernel performance because input data is not reused. A combination of vectorization and duplication achieves the highest performance of 12.3 GiB/s. Compared to an Intel Xeon E5 CPU and an Nvidia Tesla K80 GPU, the performance of the kernel on the Arria 10 FPGA is 6.7X faster than the CPU and 3X slower than the GPU. The performance per watt on the FPGA is 20.5X higher than the CPU and 1.19X lower than the GPU.

Base64 Encoding↗

DOE OSTI2024

Strain-concentration for fast, compact photonic modulation and non-volatile memory

A critical figure of merit (FoM) for electro-optic (EO) modulators is the transmission change per voltage, d T / d V . Conventional approaches in wave-guided modulators maximize d T / d V via a high EO coefficient or longer light-material interaction lengths but are ultimately limited by material losses and nonlinearities. Optical and RF resonances improve d T / d V at the cost of spectral non-uniformity, especially for high- Q optical cavity resonances. Here, we introduce an EO modulator based on piezo-strain-concentration of a photonic crystal cavity to address both trade-offs: (i) it eliminates the trade-off between d T / d V and waveguide loss—i.e., enhancement of the resonance tuning efficiency d v c / d V for the fixed EO coefficient, waveguide length, and cavity Q —and (ii) at high DC strains it exhibits a non-volatile (NV) cavity tuning Δ v c ,NV for passive memory and programming of multiple devices into resonance despite fabrication variations. The device is fabricated on a scalable silicon nitride-on-aluminum nitride platform. We measure d v c / d V =177±1MHz/V, corresponding to Δ v c =40±0.32GHz for a voltage spanning ±120V with an energy consumption of δ U /Δ v c =0.17nW/GHz. The modulation bandwidth is flat up to ω BW,3dB /2 π =3.2±0.07MHz for broadband DC-AC and 142±17MHz for resonant operation near a 2.8 GHz mechanical resonance. Optical extinction up to 25 dB is obtained via Fano-type interference. Strain-induced beam-buckling modes are programmable under a “read-write” protocol with a continuous, repeatable tuning range of 5±0.25GHz, allowing for storage and retrieval, which we quantify with mutual information of 2.4 bits and a maximum non-volatile excursion of 8 GHz. Using a full piezo-optical finite-element-model (FEM) we identify key design principles for optimizing strain-based modulators and chart a path towards achieving performance comparable to lithium niobate-based modulators and the study of high strain physics on-chip.

Wen, Y. Henry (ORCID:0009000685423628)↗

DOE OSTI2017

A Latency Tolerant Runtime System for Mass Market Computer Systems

Grappa is a latency tolerant runtime system for mass market computer systems comprised of multiple nodes having x86 multicore processors and infiniband interconnect. Grappa is designed to enable in-memory irregular applications written in the C++ programming language to scale up on multiprocessor systems. Grappa addresses the load balance, communication, and synchronization challenges posed by running applications that do not partition evenly across large systems. Two mechanisms are essential to its success: fast task switching and message aggregation. Fast task switching enables processors to tolerate communication and synchronization delays; aggregation sustains high bandwidth on low injection rate networks.

Source record↗

DOE OSTI2019

Efficient Active Message RMA in GASNet Using a Target-Side Reassembly Protocol (Extended Abstract)

GASNet is a portable, open-source, high-performance communication library designed to efficiently support the networking requirements of PGAS runtime systems and other alternative models on future exascale machines. This paper investigates strategies for efficient implementation of GASNet’s “AM Long” API that couples an RMA (Remote Memory Access) transfer with an Active Message (AM) delivery. We discuss several network-level protocols for AM Long and propose a new target-side reassembly protocol. We present a microbenchmark evaluation on the Cray XC Aries network hardware. The target-side reassembly protocol on this network improves AM Long end-to-end latency by up to 33%, and the effective bandwidth by up to 49%, while also enabling asynchronous source completion that drastically reduces injection overheads. The improved AM Long implementation for Aries is available in GASNet-EX release v2019.9.0 and later.

Hargrove, P↗

DOE OSTI2025

Deployment and Evaluation of SciStream on OLCF's Advanced Computing Ecosystem (ACE)

The growing demand for real-time analysis, experimental steering, and decision-making in scientific workflows has created a need for tightly coupled integrations between experimental facilities and high-performance computing (HPC) systems. The Department of Energy’s Integrated Research Infrastructure (IRI) initiative highlights data streaming as a key capability for enabling memory-to-memory data transfers, bypassing the limitations of traditional store-and-forward models. SciStream is a toolkit developed by researchers at Argonne National Laboratory (ANL) to support such streaming by addressing cross-domain security, delegated authentication, and application transparency. We deployed and evaluated SciStream on the Oak Ridge Leadership Computing Facility’s (OLCF) Advanced Computing Ecosystem (ACE) infrastructure, leveraging the Olivine OpenShift cluster and its high-bandwidth Data Streaming Nodes (DSNs) as gateway nodes. Our evaluation included synthetic streaming workloads derived from IRI science workflows, a streaming simulator, and integration with RabbitMQ to handle low-level messaging. This report documents the deployment process, performance evaluation, and challenges encountered, along with opportunities for future improvements.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Register files for I/O packet compression

Systems, apparatuses, and methods for reducing inter-node bandwidth are contemplated. A computer system includes requesting nodes sending transactions to target nodes. A requesting node sends a packet that includes a register identifier (ID) in place of a data value in the packet. The register ID indicates a register in the target node storing the data value. The register ID uses fewer bits in the packet than the data value. The data value may be a memory address referencing a memory location in the target node. The received packet may also include an opcode indicating an operation to perform on the targeted data value.

Roberts, David A.↗

DOE OSTI2025

Purcell-Enhanced Emissions from Diamond Color Centers in Slow Light Photonic Crystal Waveguides

Diamond color centers are promising candidates for optically addressable quantum memories, which motivates the development of efficient photonic interfaces, often using nanophotonic cavities with narrow spectral line widths and small mode volumes. However, they require perfect spectral and spatial overlap between the cavity mode and quantum emitter, which is challenging. This is especially true for solid-state quantum emitters that are often randomly positioned and suffer from inhomogeneous broadening. Another approach to enhance light-matter interaction across large optical bandwidths and areas is using slow light waveguides. Here, in this study, we demonstrate diamond slow light photonic crystal (PhC) waveguides optically coupled to embedded silicon-vacancy (SiV) color centers. We use the recently developed thin-film diamond approach to fabricate fully suspended two-dimensional PhC waveguides. We demonstrate waveguide modes with high group indices up to 70 and observe Purcell-enhanced emissions of the SiVs. Our approach represents a practical diamond platform for robust spin-photon interfaces with color centers.

2D photonic crystal waveguide↗

DOE OSTI2016

High speed radiation tolerant data links

This project was slated to design and develop Rad-Hard IP components for 1Gb/s links and supporting hardware designs such as PLL, SER/DES, pad drivers and receivers and custom protocol hardware for the 1Gb/s channel. Also included in the proposal was a study of a hardened memory to be used as a packet buffer for channel and data concentrator components to meet the 1 Gb/s specification. Over the course of the proposal, technology change and innovation of hardware designs lead us away from the 1 Gb/s goal to contemplate much higher performance link IP which, we believed better met the goals of physics experiments. Note that CERN microelectronics had managed to create a 4.7 Gb/s link designed to drive optical fibers and containing infrastructure for connecting much lower bandwidth front-end devices. Our own work to that point had shown the possibility of constructing a link with much lower power, lower physical overhead but of equivalent performance that could be designed to integrate directly onto the front-end ASIC (ADC and data encoding) designs. Substantial overall power savings and experimental simplicity could be achieved by eliminating data transmission to data concentrators and data concentrators and related hardened buffering themselves, with conversion to optical media at a removed distance from the experiment core. We had already developed and tested Rad-Hard SER/DES components (1Gb in 130nm standard cells) and redundant Pad Drivers/Receivers (3+ Gb/s designed and measured performance), and had a viable 1Gb/s link design based on redundant a stuttered clock receiver and classical PLL, so the basic goals of the proposal had been achieved. Below, in chronological order, are the products and tools we constructed, as well as our tests and publications.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2017

Merged Requests for Better Performance and Productivity in Multithreaded OpenSHMEM

A merged request is a handle representing a group of Remote Memory Access (RMA), Atomic or Collective operations. The merged request can be created either by combining multiple outstanding merged request handles or using the same merged request handle for additional operations. We show that introducing such simple yet powerful semantics in OpenSHMEM provides many productivity and performance advantages. In this paper, we first introduce the interfaces and semantics for creating and using merged request handles. Then, we demonstrate with a merge request that we can achieve better performance characteristics in multithreaded OpenSHMEM application. Particularly, we show one can achieve higher message rate, a higher bandwidth for smaller message, and better computation-communication overlap. Further, we use merged request to realize multithreaded collectives, where multiple threads co-operate to complete the collective operation. Our experimental results show that in a multithreaded OpenSHMEM program, the merged request based RMA operations achieve over 100 Million Messages Per Second (MMPS). It achieves over 10 MMPS compared to 4.5 MMPS with default RMA operations in a single threaded environment. Also, we achieve higher bandwidth for smaller message sizes, close to 100% overlap, and reduce the latency by 60%.

Gorentla Venkata, Manjunath↗