Search NASA⌕ Search

DOE OSTI · 1001595

Semantics-based distributed I/O with the ParaMEDIC framework.

Abstract

Many large-scale applications simultaneously rely on multiple resources for efficient execution. For example, such applications may require both large compute and storage resources; however, very few supercomputing centers can provide large quantities of both. Thus, data generated at the compute site oftentimes has to be moved to a remote storage site for either storage or visualization and analysis. Clearly, this is not an efficient model, especially when the two sites are distributed over a wide-area network. Thus, we present a framework called 'ParaMEDIC: Parallel Metadata Environment for Distributed I/O and Computing' which uses application-specific semantic information to convert the generated data to orders-of-magnitude smaller metadata at the compute site, transfer the metadata to the storage site, and re-process the metadata at the storage site to regenerate the output. Specifically, ParaMEDIC trades a small amount of additional computation (in the form of data post-processing) for a potentially significant reduction in data that needs to be transferred in distributed environments.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Balaji, P, Feng, W, Lin, H, Tech, Virginia. 2008-01-01. Semantics-based distributed I/O with the ParaMEDIC framework.. https://doi.org/10.1145/1383422.1383444. https://searchnasa.com/record/osti%3A1001595/semantics-based-distributed-i-o-with-the-paramedic-framework

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite Search NASA by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related reports

Fire Protection Design

The purpose of this project is to design the demolition and installation of a deluge valve swap from a DV-5 to a DV-5A valve.

99 GENERAL AND MISCELLANEOUS↗