Search NASA⌕ Search

SEARCH · Search NASA

Results for “High Performance Storage Systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

High Performance Data Analytics on a Commodity Private Cloud

Sandia is researching novel methods of analysis for large datasets. The term “Big Data” has been used frequently in recent years to describe a discipline of computer science that attempts to distil an increasing quantity of data into a result or meaningful subset that can be easily visualized and acted upon by human consumers. These data are frequently characterized by one or more of the following: high volume, velocity, variety, and veracity. Sandia scientists and researchers are no strangers to large datasets, which often typify the result of high-performance computing workloads. The challenge for Sandia data scientists, however, comes at the arrival of exascale computing, which has become mandated by executive order for national computing centers. Because of power and networking limitations, exascale computing precludes the movement of datasets at exascale from data production to data analysis. Such analysis must be done in-situ, to mitigate the time and power costs of network transport. Currently, many of Sandia’s data analytics environments exist as independent entities, often specialized for individual use cases, interconnected to other data analytics, high-performance computing, and storage systems. Sandia is procuring, installing, and researching future platforms for such data analytics. A memory footprint of 256 GB per node, fourteen data rate (FDR) InfiniBand networking, PCIe connected non-volatile memory (NVMe) data storage, and an object data store will characterize the coming institutional data analytics platforms. The combined systems will contain 300 compute nodes. A cloud-inspired software stack resulting from collaboration between the Sandia centers will allow the systems to support a flexible mix of user and system-facing workloads and use cases including code development and testing in R and Python, Apache Hadoop and Spark, distributed in-memory databases, graph analytics, emulytics, system log and predictive analytics, and data visualization for institutional and mission customers. These systems will arrive at the end of FY 2015 and production is planned by the spring of FY 2016.

97 MATHEMATICS AND COMPUTING↗

Using cross-layer adaptations for dynamic data management in large scale coupled scientific workflows, In: SC '13 Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis

As system scales and application complexity grow, managing and processing simulation data has become a significant challenge. While recent approaches based on data staging and in-situ/in-transit data processing are promising, dynamic data volumes and distributions, such as those occurring in AMR-based simulations, make the efficient use of these techniques challenging. In this paper we propose cross-layer adaptations that address these challenges and respond at runtime to dynamic data management requirements. Specifically we explore (1) adaptations of the spatial resolution at which the data is processed, (2) dynamic placement and scheduling of data processing kernels, and (3) dynamic allocation of in-transit resources. We also exploit coordinated approaches that dynamically combine these adaptations at the different layers. We evaluate the performance of our adaptive cross-layer management approach on the Intrepid IBM-BlueGene/P and Titan Cray-XK7 systems using Chombo-based AMR applications, and demonstrate its effectiveness in improving overall time-to-solution and increasing resource efficiency.

Computer Science↗

Modeling of a thermal energy storage system based on coupled metal hydrides (magnesium iron – sodium alanate) for concentrating solar power plants

Concentrating solar power plants represent low cost and efficient solutions for renewable electricity production only if adequate thermal energy storage systems are included. Metal hydride thermal energy storage systems have demonstrated the potential to achieve very high volumetric energy densities, high exergetic efficiencies, and low costs. The current work analyzes the technical feasibility and the performance of a storage system based on the high temperature Mg 2 FeH 6 hydride coupled with the low temperature Na 3 AlH 6 hydride. To accomplish this, a detailed transport model has been set up and the coupled metal hydride system has been simulated based on a laboratory scale experimental configuration. Proper kinetics expressions have been developed and included in the model to replicate the absorption and desorption process in the high temperature and low temperature hydride materials. The system showed adequate hydrogen transfer between the two metal hydrides, with almost complete charging and discharging, during both thermal energy storage and thermal energy release. The system operating temperatures varied from 450°C to 500°C, with hydrogen pressures between 30 bar and 70 bar. This makes the thermal energy storage system a suitable candidate for pairing with a solar driven steam power plant. The model results, obtained for the selected experimental configuration, showed an actual thermal energy storage system volumetric energy density of about 132 kWh/m 3 , which is more than 5 times the U.S. Department of Energy SunShot target (25 kWh/m 3 ).

14 SOLAR ENERGY↗

Best Practices and Lessons Learned from Deploying and Operating Large-Scale Data-Centric Parallel File Systems, In: SC '14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

The Oak Ridge Leadership Computing Facility (OLCF) has deployed multiple large-scale parallel file systems (PFS) to support its operations. During this process, OLCF acquired significant expertise in large-scale storage system design, file system software development, technology evaluation, benchmarking, procurement, deployment, and operational practices. Based on the lessons learned from each new PFS deployment, OLCF improved its operating procedures, and strategies. This paper provides an account of our experience and lessons learned in acquiring, deploying, and operating large-scale parallel file systems. We believe that these lessons will be useful to the wider HPC community.

Computer Science↗

Evaluation of a high-performance storage buffer with 3D XPoint devices for the DUNE data acquisition system

The DUNE detector is a neutrino physics experiment that is expected to take data starting from 2028. The data acquisition (DAQ) system of the experiment is designed to sustain several TB/s of incoming data which will be temporarily buffered while being processed by a software based data selection system. In DUNE, some rare physics processes (e.g. Supernovae Burst events) require storing the full complement of data produced over 1-2 minute window. These are recognised by the data selection system which fires a specific trigger decision. Upon reception of this decision data are moved from the temporary buffers to local, high performance, persistent storage devices. In this paper we characterize the performance of novel 3DXPoint SSD devices under different workloads suitable for high-performance storage applications. We then illustrate how such devices may be applied to the DUNE use-case: to store, upon a specific signal, 100 seconds of incoming data at 1.5 TB/s distributed among 150 identical units each operating at approximately 10GB/s.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

UnifyFS: A User-level Shared File System for Unified Access to Distributed Local Storage

We introduce UnifyFS, a user-level file system that aggregates node-local storage tiers available on high performance computing (HPC) systems and makes them available to HPC applications under a unified namespace. UnifyFS employs transparent I/O interception, so it does not require changes to application code and is compatible with commonly used HPC I/O libraries. The design of UnifyFS supports the predominant HPC I/O workloads and is optimized for bulk-synchronous I/O patterns. Furthermore, UnifyFS provides customizable file system semantics to flexibly adapt its behavior for diverse I/O workloads and storage devices. In this paper, we discuss the unique design goals and architecture of UnifyFS and evaluate its performance on a leadership-class HPC system. In our experimental results, we demonstrate that UnifyFS exhibits excellent scaling performance for write operations and can improve the performance of application checkpoint operations by as much as 3× versus a tuned configuration.

Brim, Michael↗

UPS: Unified PMU-Data Storage System to Enhance T+D PMU Data Usability

The emerging distribution-level phasor measurement unit (D-PMU) is expected to play an important role in enhancing distribution system observability and situational-awareness. It is a demanding yet challenging task to develop advanced D-PMU data management and analytics tools to improve D-PMU data usability and further promote D-PMU projects deployment. This paper focuses on D-PMU data processing and storage. It presents a brief review of existing D-PMU data storage systems and points out their limitations on high-performance, flexibility, and scalability. To overcome the limitations, a unified PMU-data storage system (UPS) is proposed. Specifically, a unified I/O interface between storage servers and computing jobs is developed to effectively reduce the overhead of managing various computing jobs and data analytics over multiple storage infrastructures; and PMUCache with PMUCache partitioning and PMUCache replacement algorithms are designed to support in-situ data processing and shared distributed data storage and further serve the computing jobs/queries with different Quality of Service (QoS) requirements. Through a series of experiments, it is demonstrated that UPS achieves high performance on fast and big data processing and storage, and efficiently increases the flexibility and scalability of PMU data management systems.

42 ENGINEERING↗

Data Management in the Continuum: Cross-facility Object-based Data Transfers

Scientific workflows are evolving from relying on a monolithic storage subsystem at a single High-Performance Computing (HPC) facility to using geographically distributed file systems, repositories, and cloud storage. As a result, storing, accessing, transferring, and managing scientific data have become highly complex and prone to performance inefficiencies. This paper delves into these challenges by exploring an optimized end-to-end interface designed to seamlessly connect various local and remote storage systems, enabling efficient data movement of objects across HPC–Cloud and HPC–HPC environments. We showcase this capability through an object-focused data management runtime system, discuss the effects of relaxed consistency semantics in distributed object scenarios, and illustrate its application in an earthquake simulation workflow. Besides reducing the amount of data by selectively transferring regions of interest, our facility-local results achieved a speedup of 45 × over an optimized HDF5 usage and 15 × over the HDF5 with caching by using the new interface in PDC-XF.

Bez, Jean Luca↗

Nature-inspired lotus-shaped fins combined with hybrid nanoparticles and metal foam for high-performance latent heat thermal energy storage

Latent heat thermal energy storage (LHTES) systems play a critical role in renewable energy integration by providing high energy density and nearly isothermal operation during phase transitions. However, their performance is often limited by slow melting/charging rates, which motivates the search for enhanced heat transfer designs. This study investigates the melting behavior of RT-82 phase change material (PCM) using novel lotus-shaped fins combined with copper metal foam and conductive graphene nanoparticles and carbon nanotubes. A two-dimensional enthalpy-porosity model in ANSYS Fluent was developed to simulate the charging/melting process, capturing non-thermal equilibrium between the foam and PCM/nano-PCM. In this study, effects of fin geometry, nanoparticle concentration, and foam porosity on melting dynamics and cost-performance trade-offs were investigated. Results showed that natural convection accelerated melting by ~12% compared to conduction-only scenarios. Optimized lotus-shaped fins with higher fin density (T3F4 and T3F10) achieved up to 63% faster melting relative to sparse configurations. Graphene nanoparticles improved thermal conductivity, with a 6% volume fraction, by reducing melting time by ~6.9%, while their combination with 75% porosity foam achieved a maximum reduction in the melting time of ~51% compared to pure PCM. Cost-performance analysis identified T3F4 as the most balanced design, offering rapid thermal response without excessive material costs, while moderate-density designs like T3S6 provided economical alternatives with acceptable performance. These results highlight the performance enhancement that can be achieved by integrating bio-inspired fins, nanoparticles, and foams, into compact and efficient LHTES for solar heating, building thermal management, and industrial waste-heat recovery applications.

25 ENERGY STORAGE↗

Geospatial Data Platform for All

Spatiotemporal data has evolved in scale due to augmented use in cross-domain applications. Simultaneously, there is substantial growth in the availability of Geographic Information Systems (GIS) data provided by the United States Geological Survey (USGS) along with other federal, state, county, or local agencies through open-data portals and public access APIs. However, data availability does not equate with accessibility. Large-scale analyses and applications require robust, performant data management with co-location of data storage and computing. The insufficiency of data management infrastructure compels researchers to adopt ad hoc project- specific GIS data storage solutions (e.g., copying data to High-Performance computer file systems). As an ad hoc storage strategy does not scale, it hampers cross-domain analyses causing difficulty in data reuse and utilizing existing code bases. Furthermore, GIS data is complex and requires expertise to analyze and manipulate due to its intricate data structures and data-specific projection transformations. Despite the challenges, we recognize that derived GIS data products, e.g., satellite or LIDAR-based images, can be used in downstream applications such as AI by domain, but non-GIS experts. To address the data needs and overcome the challenges, we are working towards a GIS Data Platform focused on efficient data storage, data discovery and access, and an API to enable common workflows. We propose a knowledge-graph (KG) approach for data discovery, whereby datasets are semantically linked to higher- level constructs such as projects and research areas. The semantic data links enable researchers to explore datasets in a top-down approach by specifying relevant and meaningful terms (assists in finding hidden data). An advantage is that the nodes and edges in a knowledge graph create built-in semantic documentation. Deeper spatiotemporal connections between data sources can be encoded via Graph Neural Networks (GNN) (Zhang et al., 2021). The KG approach can be extended to integrate the data itself in a Virtual KG (VKG). Our work will derive inspiration from large-scale VKG efforts that have been undertaken or are currently underway as part of the OpenStreetMap project (Ding et al., 2021). For DOE Data Days, we share the proposed geospatial data platform hybrid (cloud/on-prem) architecture, our work-to-date on storing, retrieving, and transforming LiDAR and raster data relevant to two important NREL use-cases, including the Renewable Energy Potential (reV) Model, and present our proposal for a KG based data discovery engine.

data platform↗

Modeling Impacts of Resilience Architectures for Extreme-Scale Storage Systems

The Scientific Modeling and Simulation of Storage project focused on improving our ability to store information reliably. The project, done jointly at the University of Wisconsin-Madison, Los Alamos National Laboratories, and Sandia National Laboratories, had the main goal of transforming the art of reliable storage system construction into a more precise science. Along those lines, the project developed numerous tools to expose problems in existing systems; these tools, when applied to the state of the art, uncovered numerous flaws in existing approaches, showing that they both are error prone and in some cases contain serious design flaws. The project also led to the creation of new approaches in reliable system construction. One such approach shows how a storage system can adapt its behavior on the fly to deliver higher performance when there are few underlying failures, but then act more conservatively when more failures are occurring to ensure the safe storage of user data. Another approach develops a storage system consistency model which both delivers high performance but high levels of reliability, a significant advance over previous approaches. Overall, the tools developed within the project advanced the science of reliable storage design, and the new approaches that resulted advanced the science of storage system construction.

97 MATHEMATICS AND COMPUTING↗

Random 3D Interpenetrating Electrode Design for Energy Storage Applications

We introduce the concept of random interpenetrating electrode structures, inspired by spinodal decomposition, as a transformative approach for energy storage applications. While ordered architectures, such as double gyroids and Kelvin cell structures, have long been favored for their structural uniformity and surface area utilization, we demonstrate that random structures offer distinct advantages including shorter diffusion pathways, higher packing density, and enhanced reaction kinetics, particularly at low temperatures. Using a combination of computational and electrochemical analysis, we show that random architecture outperforms their ordered counterparts in scenarios where diffusion dominates. This work represents the first systematic exploration of random structures in energy storage, highlighting their potential to overcome limitations of traditional designs. However, challenges such as structural reproducibility and scalability remain, necessitating further investigation into fabrication techniques and material selection. By establishing a framework to optimize random electrode architectures, we provide critical insights into the interplay between geometry, diffusion, and reaction kinetics. This study not only introduces a new design principle for 3D electrodes but also opens pathways for next-generation energy storage systems that demand both high performance and adaptability to a range of operating conditions.

Materials science↗

In situ iron coating on nanocatalysts for efficient and durable oxygen evolution reaction

The oxygen evolution reaction (OER) is essential to many important energy conversion and storage systems. To achieve high-performance and low-cost OER electrocatalysts, great effort has been devoted to the development of noble-metal-free OER nanocatalysts. While nanostructured materials have been shown effective at improving the catalytic activity, the stability of the nanocatalyst has yet to be significantly improved. Furthermore, nanocatalyst syntheses are typically complex, limiting their practical application. Herein, we design a noble-metal-free nanocatalyst which has a CoFeP x core and an in situ Fe coating layer to enhance the material’s catalytic activity and durability for OER. We utilize the vapor pressure difference of Fe and Co at high temperature to synthesize the CoFeP x -Fe core-shell nanoparticles via a one-step high-temperature shock technique. The high-temperature method strongly anchors the CoFeP x nanoparticles onto a carbon substrate. The thin Fe shell layer (~2 nm) coated on the CoFeP x greatly enhances the durability of the CoFeP x catalyst without hampering its activity due to the geometric effect. This nanocatalyst with an in situ Fe shell demonstrates a successful nanocoating approach for fabricating catalytically active protection layers on catalysts with improved activity and durability.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

BULKI-Store v0.3.2

BULKI-Store is a distributed object storage system optimized for high-performance computing environments. Built with a Rust core and Python bindings, it efficiently manages scientific and machine learning datasets across HPC clusters. The system employs a client-server architecture with MPI integration, enabling seamless scaling on supercomputers like Perlmutter. BULKI-Store's object-oriented approach provides intuitive data organization with rich metadata support, contrasting with traditional file-based solutions. Key optimizations include selective checkpoint loading, unified checkpoint files, and object chunking for large data transfers. For machine learning workloads, BULKI-Store offers advantages through fine-grained access patterns, dynamic data sharing between training instances, and reduced memory pressure. Memory management features include strategic Python GC calls, minimized data copies, and batch processing capabilities. The system leverages Rayon's thread pool for asynchronous data prefetching and supports multiple CPU architectures (ARM64, x86, AMD, RISC-V). By combining performance optimizations with developer-friendly APIs, BULKI-Store addresses the complex data management challenges of modern HPC applications while maintaining compatibility across heterogeneous computing environments.

Zhang, Wei [Lawrence Berkeley National Laboratory ↗

Vehicle Battery Safety Roadmap Guidance

The safety of electrified vehicles with high capacity energy storage devices creates challenges that must be met to assure commercial acceptance of EVs and HEVs. High performance vehicular traction energy storage systems must be intrinsically tolerant of abusive conditions: overcharge, short circuit, crush, fire exposure, overdischarge, and mechanical shock and vibration. Fail-safe responses tothese conditions must be designed into the system, at the materials and the system level, through selection of materials and safety devices that will further reduce the probability of single cell failure and preclude propagation of failure to adjacent cells. One of the most important objectives of DOE's Office of Vehicle Technologies is to support the development of lithium ion batteries thatare safe and abuse tolerant in electric drive vehicles. This Roadmap analyzes battery safety and failure modes of state-of-the-art cells and batteries and makes recommendations on future investments that would further DOE's mission.

25 ENERGY STORAGE↗

Integration experiences and performance studies of A COTS parallel archive systems

Current and future Archive Storage Systems have been asked to (a) scale to very high bandwidths, (b) scale in metadata performance, (c) support policy-based hierarchical storage management capability, (d) scale in supporting changing needs of very large data sets, (e) support standard interface, and (f) utilize commercial-off-the-shelf(COTS) hardware. Parallel file systems have been asked to do the same thing but at one or more orders of magnitude faster in performance. Archive systems continue to move closer to file systems in their design due to the need for speed and bandwidth, especially metadata searching speeds such as more caching and less robust semantics. Currently the number of extreme highly scalable parallel archive solutions is very small especially those that will move a single large striped parallel disk file onto many tapes in parallel. We believe that a hybrid storage approach of using COTS components and innovative software technology can bring new capabilities into a production environment for the HPC community much faster than the approach of creating and maintaining a complete end-to-end unique parallel archive software solution. In this paper, we relay our experience of integrating a global parallel file system and a standard backup/archive product with a very small amount of additional code to provide a scalable, parallel archive. Our solution has a high degree of overlap with current parallel archive products including (a) doing parallel movement to/from tape for a single large parallel file, (b) hierarchical storage management, (c) ILM features, (d) high volume (non-single parallel file) archives for backup/archive/content management, and (e) leveraging all free file movement tools in Linux such as copy, move, ls, tar, etc. We have successfully applied our working COTS Parallel Archive System to the current world's first petaflop/s computing system, LANL's Roadrunner, and demonstrated its capability to address requirements of future archival storage systems.

99 GENERAL AND MISCELLANEOUS↗

BurstMem: A high-performance burst buffer system for scientific applications

The growth of computing power on large-scale systems requires commensurate high-bandwidth I/O systems. Many parallel file systems are designed to provide fast sustainable I/O in response to applications' soaring requirements. To meet this need, a novel system is imperative to temporarily buffer the bursty I/O and gradually flush datasets to long-term parallel file systems. In this paper, we introduce the design of BurstMem, a high-performance burst buffer system. BurstMem provides a storage framework with efficient storage and communication management strategies. Our experiments demonstrate that BurstMem is able to speed up the I/O performance of scientific applications by up to 8.5× on leadership computer systems.

Wang, Teng↗

FCP: A Fast and Scalable Data Copy Tool for High Performance Parallel File Systems

The design of HPC file and storage systems has largely been driven by the requirements on capability, reliability, and capacity. However, the convergence of large-scale simulations with big data analytics have put the data, its usability, and management back on the front and center position. One of the most common and time consuming data management tasks is the transfer of very large datasets within and across file systems.In this paper, we are introducing the FCP tool, a file system agnostic copy tool designed at the OLCF for scalable and high-performance data transfers between two file system endpoints. It provides an array of interesting features such as adaptive chunking, checksumming on the fly, checkpoint and resume capabilities to handle failures, and preserving stripe information for Lustre file system among others. It is currently available on the Titan supercomputer at the OLCF. Initial tests have shown that FCP has much better and scalable performance than traditional data copy tools and it was capable of transferring petabyte-scale datasets between two Lustre file systems.

Wang, Feiyi↗