Search NASA⌕ Search
Home/Archive/DOE OSTI

DOE OSTI · 1095469

Using the Global Arrays Toolkit to Reimplement NumPy for Distributed Computation

Daily, Jeffrey A.·Lewis, Robert R.
View original source ↗Export metadata ↓

Abstract

Global Arrays (GA) is a software system from Pacific Northwest National Laboratory that enables an efficient, portable, and parallel shared memory programming interface to manipulate distributed defense arrays. Using a combination of GA and NumPy, we have reimplemented NumPy as a distributed drop-in replacement called Global Arrays in NumPy (GAiN). Scalability studies will be presented showing the utility of developing serial NumPy codes which can later run on more capable clusters or supercomputers.

Explore related subjects

Global ArraysNumPyPython

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Daily, Jeffrey A., Lewis, Robert R.. 2011-09-01. Using the Global Arrays Toolkit to Reimplement NumPy for Distributed Computation. https://www.osti.gov/biblio/1095469

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

DOE OSTI2013

Scalable PGAS Metadata Management on Extreme Scale Systems

Programming models intended to run on exascale systems have a number of challenges to overcome, specially the sheer size of the system as measured by the number of concurrent software entities created and managed by the underlying runtime. It is clear from the size of these systems that any state maintained by the programming model has to be strictly sub-linear in size, in order not to overwhelm memory usage with pure overhead. A principal feature of Partitioned Global Address Space (PGAS) models is providing easy access to global-view distributed data structures. In order to provide efficient access to these distributed data structures, PGAS models must keep track of metadata such as where array sections are located with respect to processes/threads running on the HPC system. As PGAS models and applications become ubiquitous on very large transpetascale systems, a key component to their performance and scalability will be efficient and judicious use of memory for model overhead (metadata) compared to application data. We present an evaluation of several strategies to manage PGAS metadata that exhibit different space/time tradeoffs. We use two real-world PGAS applications to capture metadata usage patterns and gain insight into their communication behavior.

Global Arrays↗
DOE OSTI2012

On the use of term rewriting for performance optimization of legacy HPC applications

Preparing codes for next generation supercomputer systems is anticipated to require significant changes to the optimization strategies employed in established HPC applications. In this paper, we present our experience in applying term-rewriting transformations for the optimization of such applications. We have designed application-specific term-rewriting transformations to improve the scalability, enhance locality and reduce the communication overhead of the Self-Consistent Field (SCF) computational chemistry benchmark. We present the rationale for the use of term-rewriting in this manner, the design of our transformations, and the much enhanced performance of the resulting code.

Global Arrays↗