Boost the biopharmaceutical industry’s research efficiency with hyperscale data distribution; DOE SBIR Phase I Final Technical Report
Explore the source record for details and available documents.
97 MATHEMATICS AND COMPUTING↗
Engineering topics
Fang, Chin
Publications and source records attributed to Fang, Chin.
When to use rsync
We have endeavored to show, using a series of data transfer results obtained from two testbeds, when to use the popular data copying tool rsync and related tools. Tests have been conducted in local area network (LAN) and wide area network (WAN) environments. We conclude that for files in a certain size range and network latency ≦ 10 ms round trip time (RTT), rsync is still useful for data moving tasks in the category 4 of the U.S. DOE Technical Report “Data Movement Categories”. For more demanding data movement requirements, tools of different classes are suggested. Sample histograms from two DOE user facilities are provided to further support our conclusions.
97 MATHEMATICS AND COMPUTING↗
Data Movement Categories
We have endeavored to classify the commonly seen data movement needs, as observed in data-intensive institutions (both commercial and non-profit), into four categories. Knowing how to map a data movement task into one of the four categories helps select proper data mover tools. For each category, how the data storage is involved, high-level examples and the nature of typical solutions are described. Finally, some general remarks are provided to help further orient readers new to this field - the 4th IT pillar.
99 GENERAL AND MISCELLANEOUS↗