DOE OSTI · 1351232
MarFS, a Near-POSIX Interface to Cloud Objects
Abstract
The engineering forces driving development of “cloud” storage have produced resilient, cost-effective storage systems that can scale to 100s of petabytes, with good parallel access and bandwidth. These features would make a good match for the vast storage needs of High-Performance Computing datacenters, but cloud storage gains some of its capability from its use of HTTP-style Representational State Transfer (REST) semantics, whereas most large datacenters have legacy applications that rely on POSIX file-system semantics. MarFS is an open-source project at Los Alamos National Laboratory that allows us to present cloud-style object-storage as a scalable near-POSIX file system. We have also developed a new storage architecture to improve bandwidth and scalability beyond what’s available in commodity object stores, while retaining their resilience and economy. Additionally, we present a scheme for scaling the POSIX interface to allow billions of files in a single directory and trillions of files in total.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Inman, Jeffrey Thornton, Vining, William Flynn, Ransom, Garrett Wilson, Grider, Gary Alan. 2017-01-01. MarFS, a Near-POSIX Interface to Cloud Objects. https://www.osti.gov/biblio/1351232
Cite the original work for its findings. Save a collection to share your selection of sources.