DOE OSTI · 971294
Managing high-bandwidth real-time data storage
Abstract
There exist certain systems which generate real-time data at high bandwidth, but do not necessarily require the long-term retention of that data in normal conditions. In some cases, the data may not actually be useful, and in others, there may be too much data to permanently retain in long-term storage whether it is useful or not. However, certain portions of the data may be identified as being vitally important from time to time, and must therefore be retained for further analysis or permanent storage without interrupting the ongoing collection of new data. We have developed a system, Mahanaxar, intended to address this problem. It provides quality of service guarantees for incoming real-time data streams and simultaneous access to already-recorded data on a best-effort basis utilizing any spare bandwidth. It has built in mechanisms for reliability and indexing, can scale upwards to meet increasing bandwidth requirements, and handles both small and large data elements equally well. We will show that a prototype version of this system provides better performance than a flat file (traditional filesystem) based version, particularly with regard to quality of service guarantees and hard real-time requirements.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Bigelow, David D., Brandt, Scott A, Bent, John M, Chen, Hsing-Bung. 2009-09-23. Managing high-bandwidth real-time data storage. https://www.osti.gov/biblio/971294
Cite the original work for its findings. Save a collection to share your selection of sources.