Search NASA⌕ Search

Engineering topics

Morschel, Lea

Publications and source records attributed to Morschel, Lea.

Quota management in dCache or making a perfectly normal file system normal

dCache (https://dcache.org) is a highly scalable storage system providing location-independent access to data. The data are stored across multiple data servers as complete files presented to the end-user via a single-rooted namespace. From its inception, dCache has been designed as a caching disk buffer to a tertiary tape storage system with the assumption that the latter has virtually unlimited capacity. dCache can also be configured as a disk-only storage system with no tape backend. Owing to the idea that a tape resource is infinite, or purely physically limited by budget considerations, the system has never provided for any restrictions on how much data can be stored on tape. Likewise, in the disk-only configuration, the capacity of the system is only limited by the aggregate disk capacity of the data servers. In a multi-user environment, however, this has become problematic. This presentation will describe the design and implementation of a user- and group-based quota system, that allows to manage tape and disk space allocations, as part of dCache namespace.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

dCache: from Resilience to Quality of Service

A major goal of future dCache development will be to allow users to define file Quality of Service (QoS) in a more flexible way than currently available. This will mean implementing what might be called a QoS rule engine responsible for registering and managing time-bound QoS transitions for files or storage units. In anticipation of this extension to existing dCache capabilities, the Resilience service, which maintains on-disk replica state, needs to undergo both structural modification and generalization. This paper describes ongoing work to transform Resilience into the new architecture which will eventually support a more broadly defined file QoS.

97 MATHEMATICS AND COMPUTING↗

dCache: Inter-disciplinary storage system

The dCache project provides open-source software deployed internationally to satisfy ever more demanding storage requirements. Its multifaceted approach provides an integrated way of supporting different use-cases with the same storage, from high throughput data ingest, data sharing over wide area networks, efficient access from HPC clusters and long term data persistence on a tertiary storage. Though it was originally developed for the HEP experiments, today it is used by various scientific communities, including astrophysics, biomed, life science, which have their specific requirements. In this paper we describe some of the new requirements as well as demonstrate how dCache developers are addressing them.

Mkrtchyan, Tigran↗

Improving Performance of Tape Restore Request Scheduling in the Storage System dCache

Given the anticipated increase in the amount of scientific data, it is widely accepted that primarily disk based storage will become prohibitively expensive. Tape based storage, on the other hand, provides a viable and affordable solution for the ever increasing demand for storage space. Coupled with a disk caching layer that temporarily holds a small fraction of the total data volume to allow for low latency access, it turns tape based systems into active archival storage (write once, read many) that imposes additional demands on data flow optimization compared to traditional backup setups (write once, read never). In order to preserve the lifetime of tapes and minimize the inherently higher access latency, different tape usage strategies are being evaluated. As an important disk storage system for scientific data that transparently handles tape access, dCache is making efforts to evaluate its recall optimization potential and is introducing a proof-of-concept, high-level stage request scheduling component within its SRM implementation.

Morschel, Lea↗

dCache - Keeping up With the Evolution of Science

The dCache project provides open-source software deployed internationally to satisfy ever more demanding storage requirements of various scientific communities. Its multifaceted approach provides an integrated way of supporting different use-cases with the same storage, from high throughput data ingest, through wide access and easy integration with existing systems, including event driven workflow management. With this presentation, we will show some of the recent developments that optimize data management and access to maximise the gain from stored data.

Mkrtchyan, Tigran↗

dCache – Efficient Message Encoding For Inter-Service Communication in dCache: Evaluation of Existing Serialization Protocols as a Replacement for Java Object Serialization

As a well established, large-scale distributed storage system, dCache is used to manage and serve huge amounts of data collected by high energy physics, astrophysics and photon science experiments. Based on a microservices-like architecture, dCache is built as a modular distributed system, where each component provides a different core functionality. These services communicate by passing serialized messages to each other, a core behavior whose performance properties can consequently affect the entire system.This paper compares and evaluates different data serialization protocols in computer science with the objective of replacing and improving upon Java Object Serialization (JOS), which has increasingly presented itself as no longer being sufficiently performant for encoding messages. The criteria for choosing a new framework are collected, analyzed and formalized. The primary motivation for replacing Java serialization for encoding dCache messages is increasing the general speed of message-passing and thereby reducing the round-trip time for user requests. Emphasis is also placed on schema evolution capabilities and framework usability. Approaches for generalizing (de)serialization speed and size measurements based on data structure complexity are introduced, criteria for measuring documentation, learning curve, maintainability and introduction effort are defined. Finally, several selected serialization protocols are evaluated and compared accordingly, concluding with a recommendation for a suitable JOS replacement.

Morschel, Lea↗

Efficient Message Serialization for Inter-Service Communication in dCache [Slides]

About dCache: A distributed petabyte-scale storage system for scientific data. Joint effort between DESY(2000), FNAL(2001) and NDGF(2006). Supports standard and HEP specific access protocols and authentication mechanisms. Developed for HERA and Tevatron, used for LHC and others: Belle II, LOFAR, CTA, IceCUBE, EU-XFEL, Petra3, DUNE, and many more.

97 MATHEMATICS AND COMPUTING↗