Hodson, Stephen W | Search NASA

DOE OSTI2012

Experimental Analysis of 10Gbps Transfers Over Physical and Emulated Dedicated Connections

Long haul data transfers require the optimization and balancing of performances of host and storage systems as well as network transport. An assessment of such transport methods requires a systematic generation of throughput profiles from measurements collected over different system parameters and connection lengths. We describe host and storage systems to support wide-area I/O transfers at 10 Gbps, and present measurements of memory and disk transfer throughputs over suites of physical and emulated connections of several thousands of miles. The physical connections are limited by the infrastructure and incur significant costs. The emulated connections could be of arbitrary lengths at significantly lower costs but only approximate the physical connections. We present a differential regression method to estimate the differences between the performance profiles of physical and emulated connections, and then to estimate 'physical' profiles from emulated measurements. We present a systematic analysis of wide-area memory and disk transfer throughput measurements, and establish that robust estimates of physical profiles can be generated using much less expensive emulated connections.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2011

Diagnosing Anomalous Network Performance with Confidence

Variability in network performance is a major obstacle in effectively analyzing the throughput of modern high performance computer systems. High performance interconnec- tion networks offer excellent best-case network latencies; how- ever, highly parallel applications running on parallel machines typically require consistently high levels of performance to adequately leverage the massive amounts of available computing power. Performance analysts have usually quantified network performance using traditional summary statistics that assume the observational data is sampled from a normal distribution. In our examinations of network performance, we have found this method of analysis often provides too little data to under- stand anomalous network performance. Our tool, Confidence, instead uses an empirically derived probability distribution to characterize network performance. In this paper we describe several instances where the Confidence toolkit allowed us to understand and diagnose network performance anomalies that we could not adequately explore with the simple summary statis- tics provided by traditional measurement tools. In particular, we examine a multi-modal performance scenario encountered with an Infiniband interconnection network and we explore the performance repeatability on the custom Cray SeaStar2 interconnection network after a set of software and driver updates.

Settlemyer, Bradley W↗

DOE OSTI2011

Moving Large Data Sets Over High-Performance Long Distance Networks

In this project we look at the performance characteristics of three tools used to move large data sets over dedicated long distance networking infrastructure. Although performance studies of wide area networks have been a frequent topic of interest, performance analyses have tended to focus on network latency characteristics and peak throughput using network traffic generators. In this study we instead perform an end-to-end long distance networking analysis that includes reading large data sets from a source file system and committing large data sets to a destination file system. An evaluation of end-to-end data movement is also an evaluation of the system configurations employed and the tools used to move the data. For this paper, we have built several storage platforms and connected them with a high performance long distance network configuration. We use these systems to analyze the capabilities of three data movement tools: BBcp, GridFTP, and XDD. Our studies demonstrate that existing data movement tools do not provide efficient performance levels or exercise the storage devices in their highest performance modes. We describe the device information required to achieve high levels of I/O performance and discuss how this data is applicable in use cases beyond data movement performance.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2011

Diagnosing Anomalous Network Performance with Confidence

Variability in network performance is a major obstacle in effectively analyzing the throughput of modern high performance computer systems. High performance interconnection networks offer excellent best-case network latencies; however, highly parallel applications running on parallel machines typically require consistently high levels of performance to adequately leverage the massive amounts of available computing power. Performance analysts have usually quantified network performance using traditional summary statistics that assume the observational data is sampled from a normal distribution. In our examinations of network performance, we have found this method of analysis often provides too little data to understand anomalous network performance. In particular, we examine a multi-modal performance scenario encountered with an Infiniband interconnection network and we explore the performance repeatability on the custom Cray SeaStar2 interconnection network after a set of software and driver updates.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2011

A Technique for Moving Large Data Sets over High-Performance Long Distance Networks

In this paper we look at the performance characteristics of three tools used to move large data sets over dedicated long distance networking infrastructure. Although performance studies of wide area networks have been a frequent topic of interest, performance analyses have tended to focus on network latency characteristics and peak throughput using network traffic generators. In this study we instead perform an end-to-end long distance networking analysis that includes reading large data sets from a source file system and committing the data to a remote destination file system. An evaluation of end-to-end data movement is also an evaluation of the system configurations employed and the tools used to move the data. For this paper, we have built several storage platforms and connected them with a high performance long distance network configuration. We use these systems to analyze the capabilities of three data movement tools: BBcp, GridFTP, and XDD. Our studies demonstrate that existing data movement tools do not provide efficient performance levels or exercise the storage devices in their highest performance modes.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2011

Examining Anomalous Network Performance with Confidence

Variability in network performance is a major obstacle in effectively analyzing the throughput of modern high performance computer systems. High performance interconnection networks offer excellent best-case network latencies; however, highly parallel applications running on parallel machines typically require consistently high levels of performance to adequately leverage the massive amounts of available computing power. Performance analysts have usually quantified network performance using traditional summary statistics that assume the observational data is sampled from a normal distribution. In our examinations of network performance, we have found this method of analysis often provides too little data to understand anomalous network performance. In particular, we examine a multi-modal performance scenario encountered with an Infiniband interconnection network and we explore the performance repeatability on the custom Cray SeaStar2 interconnection network after a set of software and driver updates.

99 GENERAL AND MISCELLANEOUS↗