Search NASA⌕ Search

SEARCH · Search NASA

Results for “storage cache”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Holocene cold storage practices on the eastern Snake River Plain: A risk-mitigation strategy for lean times

Previous archaeological research in southern Idaho has suggested that climate change over the past 8000 years was not dramatic enough to alter long-term subsistence practices in the region. However, recent isotopic analyses of bison remains from cold storage caves on the Snake River Plain contest this hypothesis. Our results, when examined against an archaeoclimate model, suggest that cold storage episodes coincided with drier, warmer phases that likely reduced forage and water, and thus limited the availability of bison on the open steppe. Within this context we build a risk model to illustrate how environment might have motivated cold storage behaviors. Caching bison in cold lava tubes would have mitigated both intra-annual and inter-annual food shortages under these conditions. This analysis also suggests that skeletal fat, more than meat, may have influenced the selection, transport and storage of bison carcass parts. We deciphered when and how cold storage caves which was used to provide a more comprehensive understanding of foraging behaviors in a broad range of hunting-gathering economies.

54 ENVIRONMENTAL SCIENCES↗

/Scratch as a Cache: Rethinking HPC Center Scratch Storage

To sustain emerging data-intensive scientific applications, High Performance Computing (HPC) centers invest a notable fraction of their operating budget on a specialized fast storage system, scratch space, which is designed for storing the data of currently running and soon-to-run HPC jobs. Instead, it is often used as a standard file system, wherein users arbitrarily store their data, without any consideration to the center's overall performance. To remedy this, centers periodically scan the scratch in an attempt to purge transient and stale data. This practice of supporting a cache workload using a file system and disjoint tools for staging and purging results in suboptimal use of the scratch space. In this paper, we address the above issues by proposing a new perspective, where the HPC scratch space is treated as a cache, and data population, retention, and eviction tools are integrated with scratch management. In our approach, data is moved to the scratch space only when it needed, and unneeded data is removed as soon as possible. We also design a new job-workflow-aware caching policy that leverages user-supplied hints for managing the cache. Our evaluation using three-year job logs from the Jaguar supercomputer, shows that compared to the widely-used purge approach, workflow-aware caching optimizes scratch utilization by reducing the average amount of data read by 9.3%, and by reducing job scheduling delays associated with data staging, on average, by 282.0%.

99 GENERAL AND MISCELLANEOUS↗

Data preservation at the Fermilab Tevatron

The Fermilab Tevatron collider's data-taking run ended in September 2011, yielding a dataset with rich scientific potential. The CDF and DO experiments each have nearly 9 PB of collider and simulated data stored on tape. A large computing infrastructure consisting of tape storage, disk cache, and distributed grid computing for physics analysis with the Tevatron data is present at Fermilab. The Fermilab Run II data preservation project intends to keep this analysis capability sustained through the year 2020 or beyond. To achieve this, we are implementing a system that utilizes virtualization, automated validation, and migration to new standards in both software and data storage technology as well as leveraging resources available from currently-running experiments at Fermilab. Furthermore, these efforts will provide useful lessons in ensuring long-term data access for numerous experiments throughout high-energy physics, and provide a roadmap for high-quality scientific output for years to come.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Data preservation at the Fermilab Tevatron

The Fermilab Tevatron collider's data-taking run ended in September 2011, yielding a dataset with rich scientific potential. The CDF and D0 experiments each have approximately 9 PB of collider and simulated data stored on tape. A large computing infrastructure consisting of tape storage, disk cache, and distributed grid computing for physics analysis with the Tevatron data is present at Fermilab. The Fermilab Run II data preservation project intends to keep this analysis capability sustained through the year 2020 and beyond. To achieve this goal, we have implemented a system that utilizes virtualization, automated validation, and migration to new standards in both software and data storage technology and leverages resources available from currently-running experiments at Fermilab. Lastly, these efforts have also provided useful lessons in ensuring long-term data access for numerous experiments, and enable high-quality scientific output for years to come.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Improving Data Availability for Better Access Performance: A Study on Caching Scientific Data on Distributed Workstations

Client-side data caching serves as an excellent mechanism to store and analyze the rapidly growing amount of scientific data. In our previous work, we built a distributed local cache on unreliable desktop storage contributions. This offers several desirable properties, such as performance impedance matching, improved space utilization, and high parallel I/O bandwidth. Such a low-cost, best-effort cache, however, is faced with the vagaries of storage node availability: these donated machines may be significantly less reliable than dedicated systems and cannot be controlled centrally. In this paper, we address %the tradeoffs between techniques that favor %availability or performance when it comes to cache management. the performance impact of data availability in the distributed scientific data cache setting. We then present a novel approach to storage cache management, {\em remote partial data recovery (RPDR)}. We compare our approach to two standard techniques, namely replication and erasure coding, both extended to the target caching environment. Our evaluation uses a trace-driven simulation parameterized with benchmarking results from our distributed cache prototype. The results with multiple real-world traces indicate that RPDR significantly outperforms both replication and erasure coding in many cases and overall the combination of RPDR and erasure coding yields the best performance.

99 GENERAL AND MISCELLANEOUS↗

Multi-level Hybrid Cache: Impact and Feasibility

Storage class memories, including flash, has been attracting much attention as promising candidates fitting into in today's enterprise storage systems. In particular, since the cost and performance characteristics of flash are in-between those of DRAM and hard disks, it has been considered by many studies as an secondary caching layer underneath main memory cache. However, there has been a lack of studies of correlation and interdependency between DRAM and flash caching. This paper views this problem as a special form of multi-level caching, and tries to understand the benefits of this multi-level hybrid cache hierarchy. We reveal that significant costs could be saved by using Flash to reduce the size of DRAM cache, while maintaing the same performance. We also discuss design challenges of using flash in the caching hierarchy and present potential solutions.

99 GENERAL AND MISCELLANEOUS↗

An area model for on-chip memories and its application

An area model suitable for comparing data buffers of different organizations and arbitrary sizes is described. The area model considers the supplied bandwidth of a memory cell and includes such buffer overhead as control logic, driver logic, and tag storage. The model gave less than 10 percent error when verified against real caches and register files. It is shown that, comparing caches and register files in terms of area for the same storage capacity, caches generally occupy more area per bit than register files for small caches because the overhead dominates the cache area at these sizes. For larger caches, the smaller storage cells in the cache provide a smaller total cache area per bit than the register set. Studying cache performance (traffic ratio) as a function of area, it is shown that, for small caches, direct-mapped caches perform significantly better than four-way set-associative caches and, for caches of medium areas, both direct-mapped and set-associative caches perform better than fully associative caches.

Mulder, Johannes M.↗

Robo-line storage: Low latency, high capacity storage systems over geographically distributed networks

Rapid advances in high performance computing are making possible more complete and accurate computer-based modeling of complex physical phenomena, such as weather front interactions, dynamics of chemical reactions, numerical aerodynamic analysis of airframes, and ocean-land-atmosphere interactions. Many of these 'grand challenge' applications are as demanding of the underlying storage system, in terms of their capacity and bandwidth requirements, as they are on the computational power of the processor. A global view of the Earth's ocean chlorophyll and land vegetation requires over 2 terabytes of raw satellite image data. In this paper, we describe our planned research program in high capacity, high bandwidth storage systems. The project has four overall goals. First, we will examine new methods for high capacity storage systems, made possible by low cost, small form factor magnetic and optical tape systems. Second, access to the storage system will be low latency and high bandwidth. To achieve this, we must interleave data transfer at all levels of the storage system, including devices, controllers, servers, and communications links. Latency will be reduced by extensive caching throughout the storage hierarchy. Third, we will provide effective management of a storage hierarchy, extending the techniques already developed for the Log Structured File System. Finally, we will construct a protototype high capacity file server, suitable for use on the National Research and Education Network (NREN). Such research must be a Cornerstone of any coherent program in high performance computing and communications.

Katz, Randy H.↗

Caching data from remote memories

An approach is disclosed that caches distant memories within the storage a local node. The approach provides a memory caching infrastructure that supports virtual addressing by utilizing memory in the local node as a cache of distant memories for data granules. The data granules are accessed along with metadata and an ECC associated with the data granule. The metadata is updated to indicate storage of the selected data granule in the cache.

Johns, Charles Ray↗

Evaluate data lake design for the accelerator control system

Increasing precision in automation for modern particle accelerators not only creates a requirement to gather data from all devices but also demands scalable and high-performance data infrastructure with the capability of handling vast incoming device data. A well architected data lake is suitable for such a system which integrates real-time data acquisition, transient data caching, and long-term storage. This paper evaluates data lake architecture for an Accelerator Control System (ACS), focusing on two critical components of a data lake, data cache and long-term storage.

Jaikar, Amol [Fermilab]↗

The Scalable Checkpoint/Restart Library

The Scalable Checkpoint/Restart (SCR) library provides an interface that codes may use to worite our and read in application-level checkpoints in a scalable fashion. In the current implementation, checkpoint files are cached in local storage (hard disk or RAM disk) on the compute nodes. This technique provides scalable aggregate bandwidth and uses storage resources that are fully dedicated to the job. This approach addresses the two common drawbacks of checkpointing a large-scale application to a shared parallel file system, namely, limited bandwidth and file system contention. In fact, on current platforms, SCR scales linearly with the number of compute nodes. It has been benchmarked as high as 720GB/s on 1094 nodes of Atlas, which is nearly two orders of magnitude faster thanthe parallel file system.

ECP↗

ROPEC - ROtary PErcussive Coring Drill for Mars Sample Return

The ROtary Percussive Coring Drill is a light weight, flight-like, five-actuator drilling system prototype designed to acquire core material from rock targets for the purposes of Mars Sample Return. In addition to producing rock cores for sample caching, the ROPEC drill can be integrated with a number of end effectors to perform functions such as rock surface abrasion, dust and debris removal, powder and regolith acquisition, and viewing of potential cores prior to caching. The ROPEC drill and its suite of end effectors have been demonstrated with a five degree of freedom Robotic Arm mounted to a mobility system with a prototype sample cache and bit storage station.

Chu, Philip↗

QoS-Enabled, High-performance Storage Systems for Data-Intensive Scientific Computing (Final Report)

This project's work resulted in the following research projects: (1) BORG - Block-reORGanization for Self-optimizing Storage Systems; (2) ABLE - Active Block Layer Extensions; (3) EXCES - EXternal Caching in Energy-Saving Storage Systems; and (4) GRIO - Guaranteed-Rate I/O Scheduler. These projects together help in substantially advancing the over-arching project goal of developing "QoS-Enabled, High-Performance Storage Systems".

99 GENERAL AND MISCELLANEOUS↗

Accessing Data Stored in Amazon S3 Using the Hyrax OPeNDAP Server

For three years we have been investigating data storage and retrieval in the Amazon Cloud, striving to optimize both storage structures and (cache-enhanced retrieval processes. These optimizations are hidden from data users, who simply employ the Data Access Protocol (DAP), a popular, Web-based Application Programmer Interface (API) developed by OPeNDAP. We present our collected findings and their realization in the Hyrax data server. Specific techniques include caching data on spinning disk; accessing sharded data in place; optimizing the organizational structures for data within the flat key-value space of Simple Storage Service (S3); and employing an API extension that permits simultaneous operations on many datasets in a single request.

Data access↗

Data Movement Dominates: Final Report

Over the past three years in this project, what we have observed is that the primary reason for data movement in large-scale systems is that the per-node capacity is not large enough—i.e., one of the solutions to the data-movement problem (certainly not the only solution that is required, but a significant one nonetheless) is to increase per-node capacity so that inter-node traffic is reduced. This unfortunately is not as simple as it sounds. Today’s main memory systems for datacenters, enterprise computing systems, and supercomputers, fail to provide high per-socket capacity [Dirik & Jacob 2009; Cooper-Balis et al. 2012], except at extremely high price points (factors of 10–100x the cost/bit of consumer main-memory systems) [Stokes 2008]. The reason is that our choice of technology for today’s main memory systems—i.e., DRAM, which we have used as a main-memory technology since the 1970s [Jacob et al. 2007]—can no longer keep up with our needs for density and price per bit. Main memory systems have always been built from the cheapest, densest, lowest-power memory technology available, and DRAM is no longer the cheapest, the densest, nor the lowest-power storage technology out there. It is now time for DRAM to go the way that SRAM went: move out of the way for a cheaper, slower, denser storage technology, and become a cache instead. This inflection point has happened before, in the context of SRAM yielding to DRAM. There was once a time that SRAM was the storage technology of choice for all main memories [Tomasulo 1967; Thornton 1970; Kidder 1981]. However, once DRAM hit volume production in the 1970s and 80s, it supplanted SRAM as a main memory technology because it was cheaper, and it was denser. It also happened to be lower power, but that was not the primary consideration of the day. At the time, it was recognized that DRAM was much slower than SRAM, but it was only at the supercomputer level (For instance the Cray X-MP in the 1980s and its follow-on, the Cray Y-MP, in the 1990s) that could one afford to build ever- larger main memories out of SRAM—the reasoning for moving to DRAM was that an appropriately designed memory hierarchy, built of DRAM as main memory and SRAM as a cache, would approach the performance of SRAM, at the price-per-bit of DRAM [Mashey 1999]. Today it is quite clear that, were one to build an entire multi-gigabyte main memory out of SRAM instead of DRAM, one could improve the performance of almost any computer system by up to an order of magnitude—but this option is not even considered, because to build that system would be prohibitively expensive. It is now time to revisit the same design choice in the context of modern technologies and modern systems. For reasons both technical and economic, we can no longer afford to build ever-larger main memory systems out of DRAM. Flash memory, on the other hand, is significantly cheaper and denser than DRAM and therefore should take its place. While it is true that flash is significantly slower than DRAM, one can afford to build much larger main memories out of flash than out of DRAM, and we show that an appropriately designed memory hierarchy, built of flash as main memory and DRAM as a cache, will approach the performance of DRAM, at the price-per-bit of flash. In our studies as part of this project, we have investigated Non-Volatile Main Memory (NVMM), a new main-memory architecture for large-scale computing systems, one that is specifically designed to address the weaknesses described previously. In particular, it provides the following features: non-volatility: The bulk of the storage is comprised of NAND flash, and in this organization DRAM is used only as a cache, not as main memory. Furthermore, the flash is journaled, which means that operations such as checkpoint/restore are already built into the system. 1+ terabytes of storage per socket: SSDs and DRAM DIMMs have roughly the same form factor (several square inches of PCB surface area), and terabyte SSDs are now commonplace. performance approaching that of DRAM: DRAM is used as a cache to the flash system. price-per-bit approaching that of NAND: Flash is currently well under $0.50 per gigabyte; DDR3 SDRAM is currently just over $10 per gigabyte [Newegg 2014]. Even today, one can build an easily affordable main memory system with a terabyte or more of NAND storage per CPU socket (which would be extremely expensive were one to use DRAM), and our cycle- accurate, full-system experiments show that this can be done at a performance point that lies within a factor of two of DRAM.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Lightweight storage and overlay networks for fault tolerance.

The next generation of capability-class, massively parallel processing (MPP) systems is expected to have hundreds of thousands to millions of processors, In such environments, it is critical to have fault-tolerance mechanisms, including checkpoint/restart, that scale with the size of applications and the percentage of the system on which the applications execute. For application-driven, periodic checkpoint operations, the state-of-the-art does not provide a scalable solution. For example, on today's massive-scale systems that execute applications which consume most of the memory of the employed compute nodes, checkpoint operations generate I/O that consumes nearly 80% of the total I/O usage. Motivated by this observation, this project aims to improve I/O performance for application-directed checkpoints through the use of lightweight storage architectures and overlay networks. Lightweight storage provide direct access to underlying storage devices. Overlay networks provide caching and processing capabilities in the compute-node fabric. The combination has potential to signifcantly reduce I/O overhead for large-scale applications. This report describes our combined efforts to model and understand overheads for application-directed checkpoints, as well as implementation and performance analysis of a checkpoint service that uses available compute nodes as a network cache for checkpoint operations.

99 GENERAL AND MISCELLANEOUS↗

Performance Modeling of Network-Attached Storage Device Based Hierarchical Mass Storage Systems

Network attached storage devices improve I/O performance by separating control and data paths and eliminating host intervention during the data transfer phase. Devices are attached to both a high speed network for data transfer and to a slower network for control messages. Hierarchical mass storage systems use disks to cache the most recently used files and a combination of robotic and manually mounted tapes to store the bulk of the files in the file system. This paper shows how queuing network models can be used to assess the performance of hierarchical mass storage systems that use network attached storage devices as opposed to host attached storage devices. Simulation was used to validate the model. The analytic model presented here can be used, among other things, to evaluate the protocols involved in 1/0 over network attached devices.

Menasce, Daniel A.↗

Improving Performance of Tape Restore Request Scheduling in the Storage System dCache

Given the anticipated increase in the amount of scientific data, it is widely accepted that primarily disk based storage will become prohibitively expensive. Tape based storage, on the other hand, provides a viable and affordable solution for the ever increasing demand for storage space. Coupled with a disk caching layer that temporarily holds a small fraction of the total data volume to allow for low latency access, it turns tape based systems into active archival storage (write once, read many) that imposes additional demands on data flow optimization compared to traditional backup setups (write once, read never). In order to preserve the lifetime of tapes and minimize the inherently higher access latency, different tape usage strategies are being evaluated. As an important disk storage system for scientific data that transparently handles tape access, dCache is making efforts to evaluate its recall optimization potential and is introducing a proof-of-concept, high-level stage request scheduling component within its SRM implementation.

Morschel, Lea↗