Search NASA⌕ Search

DOE OSTI · 1657070

Geomancy: Automated Performance Enhancement through Data Layout Optimization

Abstract

Large distributed storage systems such as high- performance computing (HPC) systems used by national or international laboratories require sufficient performance and scale for demanding scientific workloads and must handle shifting workloads with ease. Ideally, data is placed in locations to optimize performance, but the size and complexity of large storage systems inhibit rapid effective restructuring of data layouts to maintain performance as workloads shift. To address these issues, we have developed Geomancy, a tool that models the placement of data within a distributed storage system and reacts to drops in performance. Using a combination of machine learning techniques suitable for temporal modeling, Geomancy determines when and where a bottleneck may happen due to changing workloads and suggests changes in the layout that mitigate or prevent them. Our approach to optimizing throughput offers benefits for storage systems such as avoiding potential bottlenecks and increasing overall I/O throughput from 11% to 30%.

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Bel, Oceane M., Chang, Kenneth, Tallent, Nathan R., Duellmann, Dirk, Miller, Ethan L., Faisal Nawab, Faisal Nawab, Long, Darrell. 2020-10-29. Geomancy: Automated Performance Enhancement through Data Layout Optimization. https://searchnasa.com/record/osti%3A1657070/geomancy-automated-performance-enhancement-through-data-layout-optimization

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Search NASA by Indiciae: BibTeX · RIS.