Search NASA⌕ Search

Engineering topics

Wang, Daniel L

Publications and source records attributed to Wang, Daniel L.

Scalla: Structured Cluster Architecture for Low Latency Access

Scalla is a distributed low-latency file access system that incorporates novel techniques that minimize latency and maximize scalability over a large distributed system with a distributed namespace. Scalla's techniques have shown to be effective in nearly a decade of service for the high-energy physics community using commodity hardware and interconnects. We describe the two components used in Scalla that are instrumental in its ability to provide low-latency, fault-tolerant name resolution and load distribution, and enable its use as a high-throughput, low-latency communication layer in the Qserv system, the Large Synoptic Survey Telescope's (LSST's) prototype astronomical query system. Scalla arguably exceeded its three main design objectives: low latency, scaling, and recoverability. In retrospect, these objectives were met using a simple but effective design. Low latency was met by uniformly using linear or constant time algorithms in all high-use paths, avoiding locks whenever possible, and using compact data structures to maximize the memory caching efficiency. Scaling was achieved by architecting the system as a 64-ary tree. Nodes can be added easily and as the number of nodes increases, search performance increases at an exponential rate. Recoverability is inherent in that no permanent state information is maintained and whatever state information is needed it can be quickly constructed or reconstructed in real time. This allows dynamic changes in a cluster of servers with little impact on over-all performance or usability. Today, Scalla is being deployed in environments and for uses that were never conceived in 2001. This speaks well for the systems adaptability but the underlying reason is that the system can meet its three fundamental objectives at the same time.

99 GENERAL AND MISCELLANEOUS↗

Facts About XLDB-2011

This note provides details of the 5th Extremely Large Databases Conference and Invitational Workshop that were held in 2011 on 18-19 October and 20 October, respectively, at the SLAC National Accelerator Laboratory in Menlo Park, California. The main goals of the conference were: (1) Encourage and accelerate the exchange of ideas between users trying to build extremely large databases worldwide and database solution providers; (2) Share lessons, trends, innovations, and challenges related to building extremely large databases; (3) Facilitate the development and growth of practical technologies for extremely large databases; and (4) Strengthen, expand, and engage the XLDB community.

43 PARTICLE ACCELERATORS↗