MetallData
MetallData is an HPC platform for interactive data science applications at HPC-scales. It provides an ecosystem for persistent distributed data structures, including algorithms, interactivity and storage.
Engineering topics
Publications and source records attributed to Pearce, RogerA.
MetallData is an HPC platform for interactive data science applications at HPC-scales. It provides an ecosystem for persistent distributed data structures, including algorithms, interactivity and storage.
Privateer is a general-purpose data store that optimizes the tradeoff between storage space utilization and I/O performance. Privateer uses memory-mapped I/O with private mapping and an optimized writeback mechanism to maximize write parallelism and eliminate redundant writes; it also uses contentaddressable storage to optimize storage space via de-duplication.
TriPoll is software to perform surveys of triangles within large graphs containing metadata. In TriPoll, users define their own callback functions that get executed on the metadata of each triangle within a graph as triangles are identified. This functionality makes it possible for users to perform advanced analyses and gather statistics about specific classes of triangles of interest.
Logic in Memory Emulator (LiME) is a hardware/software tool specially designed for memory system evaluation and experiment. Emerging memories display a wide range of bandwidths, latencies, and capacities, making it challenging for the computer architect to navigate the design space of potential memory configurations, and for the application developer to assess performance implications of using such memories. With the LiME framework, architectural ideas can be prototyped in great detail yet with sufficient performance to support realistic evaluation on long running applications. LiME consists of two fundamental components: 1) the hardware and OS infrastructure for the emulator, and 2) a suite of benchmark applications to assist in characterizing the performance of current and future computer architectures. Some of the applications have been collected from other open source projects. Uses: Logging, replay and analysis of an application's memory behavior Evaluate impact of emerging memory technology on application performance. Emulate complex memory interactions in whole applications orders of magnitude faster than software simulation. Emulate acceleration hardware co-located with the memory subsystem. Features: Capture and log external memory accesses to a separate off-chip memory device without affecting application execution. Memory traces include the address, length, timestamp, and optionally the data for each transaction. Captured trace data can be saved to an SD card for off-line analysis. Configure a wide range of memory latencies in sub-nanosecond increments that encompass highbandwidth and storage class memories. Specify regions of interest (ROI) in applications to reduce the amount of trace data captured for analysis. Currently supports execution on Xilinx Zynq SoC which integrates an ARM processor with FPGA logic on a single device. Applications can be run under Linux or in bare metal mode on the ARM cores.
Metall provides simplified memory allocation interfaces for out-of-core applications that use block-addressable persistent memory devices to extend the capacity of main memory and to persistently store data on such devices. Using Metall, users can dynamically allocate memory in persistent memory devices just as in main-memory. In order to store data persistently and reload the data easily, Metall also provides a key-value store like interface. Metall offers APIs to allocate contiguous memory regions like malloc(3) does and to construct C++ objects, including STL-like containers. Metall has a similar APIs to Boost.Interprocess (BIP).