Search NASA⌕ Search

Engineering topics

Dobson, Jonathan D

Publications and source records attributed to Dobson, Jonathan D.

A Technique for Moving Large Data Sets over High-Performance Long Distance Networks

In this paper we look at the performance characteristics of three tools used to move large data sets over dedicated long distance networking infrastructure. Although performance studies of wide area networks have been a frequent topic of interest, performance analyses have tended to focus on network latency characteristics and peak throughput using network traffic generators. In this study we instead perform an end-to-end long distance networking analysis that includes reading large data sets from a source file system and committing the data to a remote destination file system. An evaluation of end-to-end data movement is also an evaluation of the system configurations employed and the tools used to move the data. For this paper, we have built several storage platforms and connected them with a high performance long distance network configuration. We use these systems to analyze the capabilities of three data movement tools: BBcp, GridFTP, and XDD. Our studies demonstrate that existing data movement tools do not provide efficient performance levels or exercise the storage devices in their highest performance modes.

99 GENERAL AND MISCELLANEOUS↗

Collecting sensor data in a high-performance computing environment: a case-study

Many research questions remain open with regard to improving reliability in exascale systems. Among others, statistics-based analysis has been used to find anomalies, to isolate root causes, and attempt to predict failures. But well-understood methods and best practices for collecting reliability data in a uniform way are still lacking, which impedes analysis. We report our experience with collecting these data from heterogeneous sources on a testbed cluster and present our data collection tool. This case illustrates the fact that reported metrics largely depend upon individual system configuration. We then investigate standards and specifications in manufacturing and desktop computing to identify concepts that may be useful for representing High Performance Computing (HPC) data and present a taxonomy that utilizes these concepts.

99 GENERAL AND MISCELLANEOUS↗