Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed Storage System”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1996

Distributed Large Data-Object Environments: End-to-End Performance Analysis of High Speed Distributed Storage Systems in Wide Area ATM Networks

We have developed and deployed a distributed-parallel storage system (DPSS) in several high speed asynchronous transfer mode (ATM) wide area networks (WAN) testbeds to support several different types of data-intensive applications. Architecturally, the DPSS is a network striped disk array, but is fairly unique in that its implementation allows applications complete freedom to determine optimal data layout, replication and/or coding redundancy strategy, security policy, and dynamic reconfiguration. In conjunction with the DPSS, we have developed a 'top-to-bottom, end-to-end' performance monitoring and analysis methodology that has allowed us to characterize all aspects of the DPSS operating in high speed ATM networks. In particular, we have run a variety of performance monitoring experiments involving the DPSS in the MAGIC testbed, which is a large scale, high speed, ATM network and we describe our experience using the monitoring methodology to identify and correct problems that limit the performance of high speed distributed applications. Finally, the DPSS is part of an overall architecture for using high speed, WAN's for enabling the routine, location independent use of large data-objects. Since this is part of the motivation for a distributed storage system, we describe this architecture.

Johnston, William↗
DOE OSTI2023

Principled Schedulability Analysis for Distributed Storage Systems Using Thread Architecture Models

In this article, we present an approach to systematically examine the schedulability of distributed storage systems, identify their scheduling problems, and enable effective scheduling in these systems. We use Thread Architecture Models (TAMs) to describe the behavior and interactions of different threads in a system, and show both how to construct TAMs for existing systems and utilize TAMs to identify critical scheduling problems. We specify three schedulability conditions that a schedulable TAM should satisfy: completeness, local enforceability, and independence; meeting these conditions enables a system to easily support different scheduling policies. We identify five common problems that prevent a system from satisfying the schedulability conditions, and show that these problems arise in existing systems such as HBase, Cassandra, MongoDB, and Riak, making it difficult or impossible to realize various scheduling disciplines. We demonstrate how to address these schedulability problems using both direct and indirect solutions, with different trade-offs. To show how to apply our approach to enable scheduling in realistic systems, we develop Tamed-HBase and Muzzled-HBase, sets of modifications to HBase that can realize the desired scheduling disciplines, including fairness and priority scheduling, even when presented with challenging workloads.

Computer Science↗
NASA NTRS1994

New architectural paradigms for multi-petabyte distributed storage systems

In the not too distant future, programs such as NASA's Earth Observing System, NSF/ARPA/NASA's Digital Libraries Initiative and Intelligence Community's (NSA, CIA, NRO, etc.) mass storage system upgrades will all require multi-petabyte (petabyte: 1015 bytes of bitfile data) (or larger) distributed storage solutions. None of these requirements, as currently defined, will meet their objectives utilizing either today's architectural paradigms or storage solutions. Radically new approaches will be required to not only store and manage veritable 'mountain ranges of data', but to make the cost of ownership affordable, much less practical in today's (and certainly the future's) austere budget environment! Within this paper we will explore new architectural paradigms and project systems performance benefits and dollars per petabyte of information stored. We will discuss essential 'top down' approaches to achieving an overall systems level performance capability sufficient to meet the challenges of these major programs.

Lee, Richard R.↗
DOE OSTI2021

Systems and methods for randomized, packet-based power management of conditionally-controlled loads and bi-directional distributed energy storage systems

The present disclosure provides a distributed and anonymous approach to demand response of an electricity system. The approach conceptualizes energy consumption and production of distributed-energy resources (DERs) via discrete energy packets that are coordinated by a cyber computing entity that grants or denies energy packet requests from the DERs. The approach leverages a condition of a DER, which is particularly useful for (1) thermostatically-controlled loads, (2) non-thermostatic conditionally-controlled loads, and (3) bi-directional distributed energy storage systems. In a first aspect of the present approach, each DER independently requests the authority to switch on for a fixed amount of time (i.e., packet duration). The coordinator determines whether to grant or deny each request based electric grid and/or energy or power market conditions. In a second aspect, bi-directional DERs, such as distributed-energy storage systems (DESSs) are further able to request to supply energy to the grid.

Frolik, Jeff↗
NASA NTRS1993

NASA Langley Research Center's distributed mass storage system

There is a trend in institutions with high performance computing and data management requirements to explore mass storage systems with peripherals directly attached to a high speed network. The Distributed Mass Storage System (DMSS) Project at NASA LaRC is building such a system and expects to put it into production use by the end of 1993. This paper presents the design of the DMSS, some experiences in its development and use, and a performance analysis of its capabilities. The special features of this system are: (1) workstation class file servers running UniTree software; (2) third party I/O; (3) HIPPI network; (4) HIPPI/IPI3 disk array systems; (5) Storage Technology Corporation (STK) ACS 4400 automatic cartridge system; (6) CRAY Research Incorporated (CRI) CRAY Y-MP and CRAY-2 clients; (7) file server redundancy provision; and (8) a transition mechanism from the existent mass storage system to the DMSS.

Pao, Juliet Z.↗
DOE OSTI2021

Branching Dueling Q-Network Based Online Scheduling of a Microgrid With Distributed Energy Storage Systems

This letter investigates a Branching Dueling Q-Network (BDQ) based online operation strategy for a microgrid with distributed battery energy storage systems (BESSs) operating under uncertainties. Additionally, the developed deep reinforcement learning (DRL) based microgrid online optimization strategy can achieve a linear increase in the number of neural network outputs with the number of distributed BESSs, which overcomes the curse of dimensionality caused by the charge and discharge decisions of multiple BESSs. Numerical simulations validate the effectiveness of the proposed method.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2004

Development of a Bio-nanobattery for Distributed Power Storage Systems

Currently available power storage systems, such as those used to supply power to microelectronic devices, typically consist of a single centralized canister and a series of wires to supply electrical power to where it is needed in a circuit. As the size of electrical circuits and components become smaller, there exists a need for a distributed power system to reduce Joule heating, wiring, and to allow autonomous operation of the various functions performed by the circuit. Our research is being conducted to develop a bio-nanobattery using ferritins reconstituted with both an iron core (Fe-ferritin) and a cobalt core (Co-ferritin). Both Co-ferritin and Fe-ferritin were synthesized and characterized as candidates for the bio-nanobattery. The reducing capability was determined as well as the half-cell electrical potentials, indicating an electrical output of nearly 0.5 V for the battery cell. Ferritins having other metallic cores are also being investigated, in order to increase the overall electrical output. Two dimensional ferritin arrays were also produced on various substrates, demonstrating the necessary building blocks for the bio-nanobattery. The bio-nanobattery will play a key role in moving to a distributed power storage system for electronic applications.

King, Glen C.↗
DOE OSTI2020

How fast can one resize a distributed file system?

Efficient resource utilization becomes a major concern as large-scale distributed computing infrastructures keep growing in size. Malleability, the possibility for resource managers to dynamically increase or decrease the amount of resources allocated to a job, is a promising way to save energy and costs. However, state-of-the-art parallel and distributed storage systems have not been designed with malleability in mind. The reason is mainly the supposedly high cost of data transfers required by resizing operations. Nevertheless, as network and storage technologies evolve, old assumptions about potential bottlenecks can be revisited. In this study, we evaluate the viability of malleability as a design principle for a distributed storage system. We specifically model the minimal duration of the commission and decommission operations. To show how our models can be used in practice, we evaluate the performance of these operations in HDFS, a relevant state-of-the-art distributed file system. We show that the existing decommission mechanism of HDFS is good when the network is the bottleneck, but can be accelerated by up to a factor 3 when storage is the limiting factor. We also show that the commission in HDFS can be substantially accelerated. With the highlights provided by our model, we suggest improvements to speed both operations in HDFS. We discuss how the proposed models can be generalized for distributed file systems with different assumptions and what perspectives are open for the design of efficient malleable distributed file systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Pufferscale: Rescaling HPC Data Services for High Energy Physics Applications

User-space HPC data services are emerging as an appealing alternative to traditional parallel file systems, because of their ability to be tailored to application needs while eliminating unnecessary overheads incurred by POSIX compliance. The High Energy Physics (HEP) community is progressively turning towards such services to enable high-throughput accesses under heavy concurrency to billions of event data produced by instruments and consumed by subsequent analysis workflows. Such services would benefit from the possibility to be rescaled up and down to adapt to changing workloads, as experimental campaigns progress, in order to optimize resource usage. This paper formalizes rescaling a distributed storage system as a multi objective optimization problem considering three criteria: load balance, data balance, and duration of the rescaling operation. We propose a heuristic for rapidly finding a good approximate solution, while allowing users to weight the criteria as needed. The heuristic is evaluated with Pufferscale, a new rescaling manager for microservice-based distributed storage systems. To validate our approach in a real-world ecosystem, we showcase the use of Pufferscale as a means to enable storage malleability in the HEPnOS storage system for HEP applications.

Distributed Storage System↗
DOE OSTI2020

Geomancy: Automated Performance Enhancement through Data Layout Optimization

Large distributed storage systems such as high- performance computing (HPC) systems used by national or international laboratories require sufficient performance and scale for demanding scientific workloads and must handle shifting workloads with ease. Ideally, data is placed in locations to optimize performance, but the size and complexity of large storage systems inhibit rapid effective restructuring of data layouts to maintain performance as workloads shift. To address these issues, we have developed Geomancy, a tool that models the placement of data within a distributed storage system and reacts to drops in performance. Using a combination of machine learning techniques suitable for temporal modeling, Geomancy determines when and where a bottleneck may happen due to changing workloads and suggests changes in the layout that mitigate or prevent them. Our approach to optimizing throughput offers benefits for storage systems such as avoiding potential bottlenecks and increasing overall I/O throughput from 11% to 30%.

Bel, Oceane M.↗
NASA NTRSDate not supplied

Hierarchical Control of Distributed Battery Energy Storage Systems in a DC Microgrid

Contents - NASA Future Power Systems - DC Microgrid with Distributed Battery Energy Storage - A simplified gateway power system - Droop Control (Primary Control) - Distributed battery units, droop control, and load sharing characteristics - Simulation results - Three-Level Hierarchical Control of Distributed Battery Units During Eclipse - Secondary Control - Objective and definition of unit control error - Implementation of secondary control - Simulation results - Tertiary Control - Schedule BU current sharing with the objective of equal SoC - Simulation results•Conclusions

Jing Zhang↗
NASA NTRS1992

NSSDC Conference on Mass Storage Systems and Technologies for Space and Earth Science Applications, volume 1

Papers and viewgraphs from the conference are presented. This conference served as a broad forum for the discussion of a number of important issues in the field of mass storage systems. Topics include magnetic disk and tape technologies, optical disks and tape, software storage and file management systems, and experiences with the use of a large, distributed storage system. The technical presentations describe, among other things, integrated mass storage systems that are expected to be available commercially. Also included is a series of presentations from Federal Government organizations and research institutions covering their mass storage requirements for the 1990's.

Kobler, Benjamin↗
NASA NTRS1992

NSSDC Conference on Mass Storage Systems and Technologies for Space and Earth Science Applications, volume 3

This report contains copies of nearly all of the technical papers and viewgraphs presented at the National Space Science Data Center (NSSDC) Conference on Mass Storage Systems and Technologies for Space and Earth Science Applications. This conference served as a broad forum for the discussion of a number of important issues in the field of mass storage systems. Topics include magnetic disk and tape technologies, optical disk and tape, software storage and file management systems, and experiences with the use of a large, distributed storage system. The technical presentations describe, among other things, integrated mass storage systems that are expected to be available commercially. Also included is a series of presentations from Federal Government organizations and research institutions covering their mass storage requirements for the 1990s.

Kobler, Ben↗
NASA NTRS1991

Proceedings of the NSSDC Conference on Mass Storage Systems and Technologies for Space and Earth Science Applications

The proceedings of the National Space Science Data Center Conference on Mass Storage Systems and Technologies for Space and Earth Science Applications held July 23 through 25, 1991 at the NASA/Goddard Space Flight Center are presented. The program includes a keynote address, invited technical papers, and selected technical presentations to provide a broad forum for the discussion of a number of important issues in the field of mass storage systems. Topics include magnetic disk and tape technologies, optical disk and tape, software storage and file management systems, and experiences with the use of a large, distributed storage system. The technical presentations describe integrated mass storage systems that are expected to be available commercially. Also included is a series of presentations from Federal Government organizations and research institutions covering their mass storage requirements for the 1990's.

Blackwell, Kim↗
NASA NTRS1992

NSSDC Conference on Mass Storage Systems and Technologies for Space and Earth Science Applications, volume 2

This report contains copies of nearly all of the technical papers and viewgraphs presented at the NSSDC Conference on Mass Storage Systems and Technologies for Space and Earth Science Application. This conference served as a broad forum for the discussion of a number of important issues in the field of mass storage systems. Topics include the following: magnetic disk and tape technologies; optical disk and tape; software storage and file management systems; and experiences with the use of a large, distributed storage system. The technical presentations describe, among other things, integrated mass storage systems that are expected to be available commercially. Also included is a series of presentations from Federal Government organizations and research institutions covering their mass storage requirements for the 1990's.

Kobler, Ben↗
NASA NTRS1993

Technology for national asset storage systems

An industry-led collaborative project, called the National Storage Laboratory, was organized to investigate technology for storage systems that will be the future repositories for our national information assets. Industry participants are IBM Federal Systems Company, Ampex Recording Systems Corporation, General Atomics DISCOS Division, IBM ADSTAR, Maximum Strategy Corporation, Network Systems Corporation, and Zitel Corporation. Industry members of the collaborative project are funding their own participation. Lawrence Livermore National Laboratory through its National Energy Research Supercomputer Center (NERSC) will participate in the project as the operational site and the provider of applications. The expected result is an evaluation of a high performance storage architecture assembled from commercially available hardware and software, with some software enhancements to meet the project's goals. It is anticipated that the integrated testbed system will represent a significant advance in the technology for distributed storage systems capable of handling gigabyte class files at gigabit-per-second data rates. The National Storage Laboratory was officially launched on 27 May 1992.

Coyne, Robert A.↗
NASA NTRSDate not supplied

Hierarchical Control of Distributed Battery Energy Storage System in a DC Microgrid

This paper presents a novel hierarchical control approach of a DC microgrid (DCMG) which is supplied by a distributed battery energy storage system (BESS). With this approach, all battery units distributed in the BESS can be controlled to discharge with accurate current sharing and state-of-charge (SoC) balancing. Similar to other hierarchical control approaches used in DCMGs, this approach consists of three levels: (1) primary control, (2) secondary control and (3) tertiary control. This work includes defining a unit control error (UCE) at the secondary control level and evaluating current sharing weights at tertiary control level. A centralized controller at secondary control level is designed to detect the UCEs of each battery unit, and to restore the average voltage of a DCMG and control battery current sharing simultaneously. The distributed battery units share the load current in a DCMG based on weights. These weights are evaluated at the tertiary control level based on battery SoCs. The approach’s effectiveness was confirmed in digital simulation tests with the same simulation model as used in the NASA AMPS Modular Hardware Emulator.

Jing Zhang↗