Search NASA⌕ Search

SEARCH · Search NASA

Results for “XT5”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Providing Runtime Clock Synchronization With Minimal Node-to-Node Time Deviation on XT4s and XT5s

We present a new high precision clock synchronization algorithm designed for large XT4 and XT5 leadership-class machines. The algorithm, which is designed to support OS noise reduction through co-scheduling, is suitable for usage cases requiring low overhead and minimal time deviation between nodes. Unlike most high-precision algorithms which reach their precision in a post-mortem analysis after the application has completed, the new ORNL developed algorithm rapidly provides precise results during runtime. Previous to our work, the leading high-precision clock synchronization algorithms that made results available during runtime relied on probabilistic schemes that are not guaranteed to result in an answer.

99 GENERAL AND MISCELLANEOUS↗

Linux Kernel Co-Scheduling For Bulk Synchronous Parallel Applications

This paper describes a kernel scheduling algorithm that is based on co-scheduling principles and that is intended for parallel applications running on 1000 cores or more where inter-node scalability is key. Experimental results for a Linux implementation on a Cray XT5 machine are presented.1 The results indicate that Linux is a suitable operating system for this new scheduling scheme, and that this design provides a dramatic improvement in scaling performance for synchronizing collective operations at scale.

99 GENERAL AND MISCELLANEOUS↗

Characterizing Application Runtime Behavior from System Logs and Metrics

Large-scale systems are heavily shared resource environments where a mix of applications are launched concurrently competing for network and storage resources. It is essential to characterize the runtime behavior of these applications for provisioning system resources and understanding the impact of application s performance when competing for resources. In this paper, we study the use of zero- and low-overhead system logs and other system metric data for characterizing the runtime behavior of several applications. We present our preliminary work on estimating individual application s I/O demands by observing file system usage pattern over multiple runs, and interpreting application s network utilization characteristics by observing link-layer error logs. We also present preliminary findings on using such information in making context-sensitive scheduling decisions that minimize potentially negative interactions between applications competing for shared resources. Our analysis is based on four months of system log data collected on one of the world s largest supercomputing facilities, the Jaguar XT5 petaflop system at Oak Ridge National Laboratory.

99 GENERAL AND MISCELLANEOUS↗

Improving Multi-Million Virtual Rank MPI Execution in

(MUPI) is a parallel discrete event simulator designed for enabling software-based experimentation via simulated execution across a range of synthetic to unmodified parallel programs using the Message Passing Interface (MPI) with millions of tasks. Here, we report work in progress in improving the efficiency of . Among the issues uncovered are the scaling problems with implementing barriers and intertask message ordering. Preliminary performance shows the possibility of supporting hundreds of virtual MPI ranks per real processor core. Performance improvements of at least 2 are observed, and enable execution of benchmark MPI runs with over 16 million virtual ranks synchronized in a discrete event fashion on as few as 16,128 real cores of a Cray XT5.

99 GENERAL AND MISCELLANEOUS↗

Impact of Quad-core Cray XT4 System and Software Stack on Scientific Computation

An upgrade from dual-core to quad-core AMD processor on the Cray XT system at the Oak Ridge National Laboratory (ORNL) Leadership Computing Facility (LCF) has resulted in significant changes in the hardware and software stack, including a deeper memory hierarchy, SIMD instructions and a multi-core aware MPI library. In this paper, we evaluate impact of a subset of these key changes on large-scale scientific applications. We will provide insights into application tuning and optimization process and report on how different strategies yield varying rates of successes and failures across different application domains. For instance, we demonstrate that the vectorization instructions (SSE) provide a performance boost of as much as 50% on fusion and combustion applications. Moreover, we reveal how the resource contentions could limit the achievable performance and provide insights into how application could exploit Petascale XT5 system's hierarchical parallelism.

99 GENERAL AND MISCELLANEOUS↗

Three-Dimensional Full Core Power Calculations for Pressurized Water Reactors

We have implemented a new multilevel parallel decomposition in the Denovo discrete ordinates radiation transport code. In concert with Krylov subspace iterative solvers, the multilevel decomposition allows concurrency over energy in addition to space-angle. The original space-angle partitioning in Denovo placed an eective limit on the scalability of the transport solver that was highly dependent on the problem size. The added phase-space concurrency combined with the high-performance Krylov solvers has enabled weak scaling to 100K cores on the Jaguar XT5 supercomputer. Furthermore, the multilevel decomposition provides enough concurrency to scale to exascale computing and beyond.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

GVT Algorithms and Discrete Event Dynamics on 128K+ Processor Cores

Parallel discrete event simulation (PDES) represents a class of codes that are challenging to scale to large number of processors due to tight global timestamp-ordering and fine-grained event execution. One of the critical factors in scaling PDES is the efficiency of the underlying global virtual time (GVT) algorithm needed for correctness of parallel execution and speed of progress. Although many GVT algorithms have been proposed previously, few have been proposed for scalable asynchronous execution and none customized to exploit one-sided communication. Moreover, the detailed performance effects of actual GVT algorithm implementations on large platforms are unknown. Here, three major GVT algorithms intended for scalable execution on high-performance systems are studied: (1) a synchronous GVT algorithm that affords ease of implementation, (2) an asynchronous GVT algorithm that is more complex to implement but can relieve blocking latencies, and (3) a variant of the asynchronous GVT algorithm, proposed and studied for the first time here, to exploit one-sided communication in extant supercomputing platforms. Performance results are presented of implementations of these algorithms on over 64,000 cores of a Cray XT5 system, exercised on a range of parameters: optimistic and conservative synchronization, fine- to medium-grained event computation, synthetic and non-synthetic applications, and different lookahead values. Performance of tens of billions of events executed per second are registered, exceeding the speeds of any known PDES engine, and showing asynchronous GVT algorithms to outperform state-of-the-art synchronous GVT algorithms. Detailed PDES-specific runtime metrics are presented to further the understanding of tightly-coupled discrete event execution dynamics on massively parallel platforms.

99 GENERAL AND MISCELLANEOUS↗

Cheetah: A Framework for Scalable Hierarchical Collective Operations

Collective communication operations, used by many scientific applications, tend to limit overall parallel application performance and scalability. Computer systems are becoming more heterogeneous with increasing node and core-per-node counts. Also, a growing number of data-access mechanisms, of varying characteristics, are supported within a single computer system. We describe a new hierarchical collective communication framework that takes advantage of hardware-specific data-access mechanisms. It is flexible, with run-time hierarchy specification, and sharing of collective communication primitives between collective algorithms. Data buffers are shared between levels in the hierarchy reducing collective communication management overhead. We have implemented several versions of the Message Passing Interface (MPI) collective operations, MPI Barrier() and MPI Bcast(), and run experiments using up to 49, 152 processes on a Cray XT5, and a small InfiniBand based cluster. At 49, 152 processes our barrier implementation outperforms the optimized native implementation by 75%. 32 Byte and one Mega-Byte broadcasts outperform it by 62% and 11%, respectively, with better scalability characteristics. Improvements relative to the default Open MPI implementation are much larger.

99 GENERAL AND MISCELLANEOUS↗

Discrete Event Modeling and Massively Parallel Execution of Epidemic Outbreak Phenomena

In complex phenomena such as epidemiological outbreaks, the intensity of inherent feedback effects and the significant role of transients in the dynamics make simulation the only effective method for proactive, reactive or post-facto analysis. The spatial scale, runtime speed, and behavioral detail needed in detailed simulations of epidemic outbreaks make it necessary to use large-scale parallel processing. Here, an optimistic parallel execution of a new discrete event formulation of a reaction-diffusion simulation model of epidemic propagation is presented to facilitate in dramatically increasing the fidelity and speed by which epidemiological simulations can be performed. Rollback support needed during optimistic parallel execution is achieved by combining reverse computation with a small amount of incremental state saving. Parallel speedup of over 5,500 and other runtime performance metrics of the system are observed with weak-scaling execution on a small (8,192-core) Blue Gene / P system, while scalability with a weak-scaling speedup of over 10,000 is demonstrated on 65,536 cores of a large Cray XT5 system. Scenarios representing large population sizes exceeding several hundreds of millions of individuals in the largest cases are successfully exercised to verify model scalability.

99 GENERAL AND MISCELLANEOUS↗

Full Core Reactor Analysis: Running Denovo on Jaguar

Fully-consistent, full-core, 3D, deterministic simulations using the orthogonal mesh code Denovo were run on the massively parallel computing architecture Jaguar XT5. Using energy and spatial parallelization schemes, Denovo was able to efficiently scale to over 160k processors. Cell-homogenized cross-sections were used with Step-Characteristics, Linear-Discontinuous Finite Element, and Tri-Linear-Discontinuous Finite Element spatial methods. It was determined that using the finite element methods gave considerably more accurate eigenvalue solutions for large aspect ratios meshes than those using Step-Characteristics.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Lustre at Petascale: Experiences in Troubleshooting and Upgrading

Some veterans in the HPC industry semi-facetiously define supercomputers as devices that convert compute-bound problems into I/O-bound problems. Effective utilization of large high performance computing resources often requires access to large amounts of fast storage. The National Institute for Computational Sciences (NICS) operates Kraken, a 1.17 PetaFLOPS Cray XT5 for the National Science Foundation (NSF). Kraken s primary file system has migrated from Lustre 1.6 to 1.8 and is currently being moved to servers external to the machine. Additional bandwidth will be made available by mounting the NICS-wide Lustre file system. Newer versions of Lustre, beyond what Cray provides, are under evaluation for stability and performance. Over the past several years of operation, Kraken s Lustre file system has evolved to be extremely stable in an effort to better serve Kraken s users.

99 GENERAL AND MISCELLANEOUS↗

A Next-Generation Parallel File System Environment for the OLCF

When deployed in 2008/2009 the Spider system at the Oak Ridge National Laboratory s Leadership Computing Facility (OLCF) was the world s largest scale Lustre parallel file system. Envisioned as a shared parallel file system capable of delivering both the bandwidth and capacity requirements of the OLCF s diverse computational environment, Spider has since become a blueprint for shared Lustre environments deployed worldwide. Designed to support the parallel I/O requirements of the Jaguar XT5 system and other smallerscale platforms at the OLCF, the upgrade to the Titan XK6 heterogeneous system will begin to push the limits of Spider s original design by mid 2013. With a doubling in total system memory and a 10x increase in FLOPS, Titan will require both higher bandwidth and larger total capacity. Our goal is to provide a 4x increase in total I/O bandwidth from over 240GB=sec today to 1TB=sec and a doubling in total capacity. While aggregate bandwidth and total capacity remain important capabilities, an equally important goal in our efforts is dramatically increasing metadata performance, currently the Achilles heel of parallel file systems at leadership. We present in this paper an analysis of our current I/O workloads, our operational experiences with the Spider parallel file systems, the high-level design of our Spider upgrade, and our efforts in developing benchmarks that synthesize our performance requirements based on our workload characterization studies.

99 GENERAL AND MISCELLANEOUS↗

Performance of the Community Earth System Model

The Community Earth System Model (CESM), released in June 2010, incorporates new physical process and new numerical algorithm options, significantly enhancing simulation capabilities over its predecessor, the June 2004 release of the Community Climate System Model. CESM also includes enhanced performance tuning options and performance portability capabilities. This paper describes the performance engineering aspects of the CESM and reports performance and performance scaling on both the Cray XT5 and the IBM BG/P for four representative production simulations, varying both problem size and enabled physical processes. The paper also describes preliminary performance results for high resolution simulations using over 200,000 processor cores, indicating the promise of ongoing work in numerical algorithms and where further work is required.

99 GENERAL AND MISCELLANEOUS↗

Design and Implementation of Broadcast Algorithms for Extreme-Scale Systems

The scalability and performance of collective communication operations limit the scalability and performance of many scientific applications. This paper presents two new blocking and nonblocking Broadcast algorithms for communicators with arbitrary communication topology, and studies their performance. These algorithms benefit from increased concurrency and a reduced memory footprint, making them suitable for use on large-scale systems. Measuring small, medium, and large data Broadcasts on a Cray-XT5, using 24,576 MPI processes, the Cheetah algorithms outperform the native MPI on that system by 51%, 69%, and 9%, respectively, at the same process count. These results demonstrate an algorithmic approach to the implementation of the important class of collective communications, which is high performing, scalable, and also uses resources in a scalable manner.

99 GENERAL AND MISCELLANEOUS↗

Integrated Radiation Transport and Thermo-Mechanics Simulation of a PWR Assembly

The Advanced Multi-Physics (AMP) Nuclear Fuel Performance code (AMPFuel) is focused on predicting the temperature and strain within a nuclear fuel assembly to evaluate the performance and safety of existing and advanced nuclear fuel bundles within existing and advanced nuclear reactors. AMPFuel was extended to include an integrated nuclear fuel assembly capability for (one-way) coupled radiation transport and nuclear fuel assembly thermo-mechanics. This capability is the initial step towards incorporating an improved predictive nuclear fuel assembly modeling capability to accurately account for source terms, such as the neutron flux distribution, coolant conditions, and assembly mechanical stresses, of traditional (single-pin) nuclear fuel performance simulation. AMPFuel was used to model an entire 17 x 17 Pressurized Water Reactor (PWR) fuel assembly with many of the features resolved in three dimensions (for thermo-mechanics and/or neutronics), including the fuel, gap, and cladding of each of the 264 fuel pins, the 25 guide tubes, top and bottom structural regions, and the upper and lower (neutron) reflector regions. The final full-assembly calculation was executed on Jaguar (Cray XT5) at the Oak Ridge Leadership Computing Facility using 40,000 cores in under 10 hours to model over 162 billion degrees of freedom for 10 loading steps.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Real-Time System Log Monitoring/Analytics Framework

Analyzing system logs provides useful insights for identifying system/application anomalies and helps in better usage of system resources. Nevertheless, it is simply not practical to scan through the raw log messages on a regular basis for large-scale systems. First, the sheer volume of unstructured log messages affects the readability, and secondly correlating the log messages to system events is a daunting task. These factors limit large-scale system logs primarily for generating alerts on known system events, and post-mortem diagnosis for identifying previously unknown system events that impacted the systems performance. In this paper, we describe a log monitoring framework that enables prompt analysis of system events in real-time. Our web-based framework provides a summarized view of console, netwatch, consumer, and apsched logs in real- time. The logs are parsed and processed to generate views of applications, message types, individual/group of compute nodes, and in sections of the compute platform. Also from past application runs we build a statistical profile of user/application characteristics with respect to known system events, recoverable/non-recoverable error messages and resources utilized. The web-based tool is being developed for Jaguar XT5 at the Oak Ridge Leadership Computing facility.

Oral, H Sarp↗

A Tale of Two Systems: Flexibility of Usage of Kraken and Nautilus at the National Institute for Computational Sciences

The National Institute for Computational Sciences (NICS) at the University of Tennessee currently operates two computational resources for the eXtreme Science and Engineering Discovery Environment (XSEDE), Kraken, a 112,896-core Cray XT5 for general purpose computation, and Nautilus, a 1,024-core SGI Altix UV 1000 for data analysis and visualization. We analyze a year's worth of accounting logs for Kraken and Nautilus to understand how users take advantage of these two systems and how analysis jobs differ from general HPC computation We find that researchers take advantage of the flexibility offered by these systems, running a wide variety of jobs at many scales and using the full range of core counts and available memory for their jobs. The jobs on Nautilus tend to use less walltime and more memory per core than the jobs run on Kraken. Additionally, researchers are more likely to run interactive jobs on Nautilus than on Kraken. Small jobs experience a good quality of service on both systems. This information can be used for the management and allocation of time on existing HPC and analysis systems as well as for planning for deploying future HPC and analysis systems.

Szczepanski, Amy F↗

Simulating Billion-Task Parallel Programs

In simulating large parallel systems, bottom-up approaches exercise detailed hardware models with effects from simplified software models or traces, whereas top-down approaches evaluate the timing and functionality of detailed software models over coarse hardware models. Here, we focus on the top-down approach and significantly advance the scale of the simulated parallel programs. Via the direct execution technique combined with parallel discrete event simulation, we stretch the limits of the top-down approach by simulating message passing interface (MPI) programs with millions of tasks. Using a timing-validated benchmark application, a proof-of-concept scaling level is achieved to over 0.22 billion virtual MPI processes on 216,000 cores of a Cray XT5 supercomputer, representing one of the largest direct execution simulations to date, combined with a multiplexing ratio of 1024 simulated tasks per real task.

Perumalla, Kalyan S↗