Workload-adaptive Scheduling for Efficient Use of Parallel File Systems in High-performance Computing Clusters
Explore the source record for details and available documents.
Goponenko, Alexander V.↗
SEARCH · Search NASA
Results for “high performance computing systems”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Preemptive resource management for dynamically arriving tasks in an oversubscribed heterogeneous computing system
We design resource management heuristics that assign serial tasks to the nodes of a heterogeneous high performance computing (HPC) system. The value of completing these tasks is modeled using monotonically decreasing utility functions that represent the time-varying importance of the task. The value of completing a task is equal to its utility function at the time of its completion. The overall performance of this system is measured using the total utility earned by all tasks during some interval of time. To maximize the performance of such a system where the preemption of tasks is possible, we have designed, analyzed, and compared a set of resource allocation heuristic techniques. We combine two utility-aware heuristics with three different preemption techniques to create six preemption-capable heuristics. We also consider the two utility-aware heuristics without preemption. We use simulation studies to evaluate this set of eight heuristics and compare them with an FCFS heuristic, which is often used in real systems, and random assignments. In general, our set of eight heuristics is able to significantly outperform the comparison heuristics, and the preemption-capable heuristics are able to significantly increase the utility earned compared to the heuristics that do not use preemption. We analyze the performance tradeoffs among the different preemption-capable heuristics under a variety of oversubscribed workload environments.
97 MATHEMATICS AND COMPUTING↗
Failure prediction using machine learning in a virtualised HPC system and application
Failure is an increasingly important issue in high performance computing and cloud systems. As large-scale systems continue to grow in scale and complexity, mitigating the impact of failure and providing accurate predictions with sufficient lead time remains a challenging research problem. Traditional existing fault-tolerance strategies such as regular check-pointing and replication are not adequate because of the emerging complexities of high performance computing systems. This necessitates the importance of having an effective as well as proactive failure management approach in place aimed at minimizing the effect of failure within the system. With the advent of machine learning techniques, the ability to learn from past information to predict future pattern of behaviours makes it possible to predict potential system failure more accurately. Thus, in this paper, we explore the predictive abilities of machine learning by applying a number of algorithms to improve the accuracy of failure prediction. We have developed a failure prediction model using time series and machine learning, and performed comparison based tests on the prediction accuracy. The primary algorithms we considered are the support vector machine (SVM), random forest (RF), k-nearest neighbors (KNN), classification and regression trees (CART) and linear discriminant analysis (LDA). Experimental results indicates that the average prediction accuracy of our model using SVM when predicting failure is 90% accurate and effective compared to other algorithms. This finding implies that our method can effectively predict all possible future system and application failures within the system.
Mohammed, Bashir↗
Accessing protected data by a high-performance computing cluster
A data protection system is provided that allows applications to access protected data in a way that restricts applications from outputting to unauthorized targets any unprotected data derived from the protected data and that ensures that the applications do not have access to a key that allows access to the unprotected data. The data protection system provides a policy server that may execute on a service node of a high performance computing system and a data encryption process that may execute on each compute node that is allocated to an application or batch job. The policy server maintains policies of entities specifying access control for protected data. The data encryption process generates a secure execution environment for an application process and interfaces with the policy server to retrieve keys for decrypting protected data in accordance with a policy, and it decrypts and provides the decrypted data to the application process.
Barnes, Peter↗
Ensemble Simulations on Leadership Computing Systems
Scientific productivity can be enhanced through workflow management tools, relieving large High Performance Computing (HPC) system users from the tedious tasks of scheduling and designing the complex computational execution of scientific applications. This paper presents a study on the usage of ensemble workflow tools to accelerate science using the Summit and Frontier supercomputing systems. The research aims to connect science domain simulations using Oak Ridge Leadership Computing Facility (OLCF) supercomputing platforms with ensemble workflow methods in order to accelerate HPC-enabled discovery and boost scientific impact. We present the coupling, porting and optimization of Radical-Cybertools on three applications: Chroma, NAMD and LAMMPS. The tools augment traditional HPC monolithic runs with a pilot scheduler. Lessons-learned are discussed for physics, biology and materials science applications. We discuss intrinsic limitations of coupling and porting ensemble workflow tools to applications that run on large HPC systems. The origins of technical challenges and their solutions developed during the implementation process are discussed. Data management strategies, OLCF’s policies for ensembles, and natively supported workflow tools are also summarized.
Georgiadou, Antigoni [ORNL] (ORCID:000000020977631↗
DeepHyper: Asynchronous Hyperparameter Search for Deep Neural Networks
Hyperparameters employed by deep learning (DL) methods play a substantial role in the performance and reliability of these methods in practice. Unfortunately, finding performance-optimizing hyperparameter settings is a notoriously difficult task. Hyperparameter search methods typically have limited production-strength implementations or do not target scalability within a highly parallel machine, portability across different machines, experimental comparison between different methods, and tighter integration with workflow systems. In this paper, we present DeepHyper, a Python package that provides a common interface for the implementation and study of scalable hyperparameter search methods. It adopts the Balsam workflow system to hide the complexities of running large numbers of hyperparameter configurations in parallel on high-performance computing (HPC) systems. We implement and study asynchronous model-based search methods that consist of sampling a small number of input hyperparameter configurations and progressively fitting surrogate models over the input-output space until exhausting a user-defined budget of evaluations. We evaluate the efficacy of these methods relative to approaches such as random search, genetic algorithms, Bayesian optimization, and hyperband on DL benchmarks on CPU- and GPU-based HPC systems.
Balaprakash, Prasanna↗
Energy Performance Evaluation of Aquila's Aquarius Fixed Cold Plate Cooling System at NREL's High Performance Computing Center
In the first half of 2018, as part of a partnership with Sandia National Laboratories, Aquila installed its fixed cold plate, liquid-cooled Aquarius rack solution for high-performance computing (HPC) clustering at the National Renewable Energy Laboratory's (NREL's) Energy Systems Integration Facility (ESIF). This new fixed cold plate, warm-water cooling technology together with a manifold design provides easy access to service nodes and eliminates the need for server auxiliary fans altogether. Aquila and Sandia National Laboratories chose NREL's HPC Data Center for the initial installation and evaluation because the data center is configured for liquid cooling and has the required instrumentation to measure flow and temperature differences to facilitate testing. This paper gives an overview of the Aquarius fixed cold plate cooling technology and provides results from early energy performance evaluation testing.
97 MATHEMATICS AND COMPUTING↗
Characterizing the Impact of GPU Power Management on an Exascale System
As GPU-accelerated high-performance computing (HPC) systems approach exascale performance, controlling energy consumption without compromising throughput is essential. Architectures such as the AMD MI250X-based Frontier supercomputer provide runtime mechanisms like frequency and power capping, enabling energy tuning without modifying application code. Although both target energy reduction, they operate via distinct hardware control paths and influence workloads differently. We present a comprehensive evaluation of these strategies on a leadership-class system using diverse HPC proxy applications representative of production workloads. Our study analyzes performance–energy trade-offs across multiple capping levels, node counts (1 and 32), and application profiles. Results show that frequency capping generally achieves higher energy efficiency and scalability, with gains of up to 13.2% without performance loss, while power capping is more effective for single-node runs or bursty GPU utilization. We also provide practical guidelines to help system administrators and users balance energy efficiency and performance in large-scale scientific workloads.
Costa, Mariana [Universidade Federal do Rio Grande↗
Ion Coulomb Crystals in Storage Rings for Quantum Information Science
Quantum information science is a growing field that promises to take computing into a new age of higher performance and larger scale computing as well as being capable of solving problems classical computers are incapable of solving. The outstanding issue in practical quantum computing today is scaling up the system while maintaining interconnectivity of the qubits and low error rates in qubit operations to be able to implement error correction and fault-tolerant operations. Trapped ion qubits offer long coherence times that allow error correction. However, error correction algorithms require large numbers of qubits to work properly. We can potentially create many thousands (or more) of qubits with long coherence states in a storage ring. For example, a circular radio-frequency quadrupole, which acts as a large circular ion trap and could enable larger scale quantum computing. Such a Storage Ring Quantum Computer (SRQC) would be a scalable and fault tolerant quantum information system, composed of qubits with very long coherence lifetimes. With computing demands potentially outpacing the supply of high-performance systems, quantum computing could bring innovation and scientific advances to particle physics and other DOE supported programs. Increased support of R$\&$D in large scale ion trap quantum computers would allow the timely exploration of this exciting new scalable quantum computer. The R$\&$D program could start immediately at existing facilities and would include the design and construction of a prototype SRQC. We invite feedback from and collaboration with the particle physics and quantum information science communities.
43 PARTICLE ACCELERATORS↗
Sandia’s Liquid-Cooled Data Center Boosts Efficiency and Resiliency
The Federal Energy Management Program (FEMP) encourages federal agencies and organizations to improve data center energy efficiency, which can offer tremendous opportunities for energy and cost savings. In this success story, a novel liquid cooling system provides reliable, resilient, and energy-efficient cooling for high performance computing (HPC) systems at Sandia National Laboratories.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Big Data Meets HPC Log Analytics: Scalable Approach to Understanding Systems at Extreme Scale
Today's high-performance computing (HPC) systems are heavily instrumented, generating logs containing information about abnormal events, such as critical conditions, faults, errors and failures, system resource utilization, and about the resource usage of user applications. These logs, once fully analyzed and correlated, can produce detailed information about the system health, root causes of failures, and analyze an application's interactions with the system, providing valuable insights to domain scientists and system administrators. However, processing HPC logs requires a deep understanding of hardware and software components at multiple layers of the system stack. Moreover, most log data is unstructured and voluminous, making it more difficult for system users and administrators to manually inspect the data. With rapid increases in the scale and complexity of HPC systems, log data processing is becoming a big data challenge. This paper introduces a HPC log data analytics framework that is based on a distributed NoSQL database technology, which provides scalability and high availability, and the Apache Spark framework for rapid in-memory processing of the log data. The analytics framework enables the extraction of a range of information about the system so that system administrators and end users alike can obtain necessary insights for their specific needs. We describe our experience with using this framework to glean insights from the log data about system behavior from the Titan supercomputer at the Oak Ridge National Laboratory.
Park, Byung↗
HIPLZ: Enabling performance portability for exascale systems
While heterogeneous computing has emerged as a dominant trend in current and future High-Performance Computing (HPC) systems, it is also widely recognized that this shift has led to increased software complexity due to a proliferation of programming systems for different heterogeneous processors. One such example is the Heterogeneous-Compute Interface for Portability from AMD (HIP ), which is composed of a C Runtime API and C++ Kernel Language. Many HPC applications will likely use HIP on future exascale systems (e.g., Frontier and El Capitan), but HIP currently only targets AMD and NVIDIA processors. This limitation creates challenges for users who would also like to run their applications on exascale systems based on other architectures (e.g., Aurora, which is based on Intel hardware) that are currently not targeted by HIP . In this paper, we introduce the design and implementation of HIPLZ , a compiler and runtime system that uses the Intel Level Zero API to support HIP on Intel GPU architectures. We discuss the design of HIPLZ , derived from HIPCL (an implementation of HIP on top of OpenCL ), and portability issues that occur from using the Level Zero runtime as a backend. We evaluate our implementation by running several performance benchmarks and mini-apps written in HIP on Intel architectures using HIPLZ . Our results show that this approach provides competitive performance relative to Intel's OpenCL implementations on Intel Gen9 and UHD Graphics 770 GPUs, while providing good coverage of features needed by HPC applications. Overall, this approach is a promising demonstration of enabling performance portability for exascale systems.
97 MATHEMATICS AND COMPUTING↗
Frontier Job-Centric Telemetry Dataset
Comprehensive analysis of high-performance computing (HPC) systems requires linking workload execution to system behavior. This kind of analysis is vital for diagnosing performance issues, managing capacity, detecting anomalous workloads, and understanding how applications interact with system hardware. This job-centric telemetry dataset unifies scheduler job records with node-level measurements, enabling direct association between workloads and their corresponding power, thermal, and performance characteristics. It contains sanitized, scheduler related metadata for 152,400 individual jobs that ran on the Frontier supercomputer and ended on selected days throughout 2024 and 2025, a subpopulation of ~6.8% of the total number of allocated jobs with non-zero run time on the system over that same period. Each is linked with files that contain telemetry time series records of the power utilization and temperature behavior of its allocated nodes and their processors during the run time of the job. Where available, a portion of the job files also contain network performance time series. Jobs are sampled from select days that reflect normal levels of user activity and possess job size distributions with large numbers of leadership class jobs (>20% of Frontier nodes). Jobs in this dataset attempt to best represent successful user workflows.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
Sandia National Laboratories' Holistic Data Center Design Integrates Energy- and Water-Efficiency, Flexibility, and Resilience
This success story highlights Sandia National Laboratories and their holistic data center approach, which provides resilient energy- and water-efficient cooling for high-performance computing (HPC) systems.
ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION,↗
Hardware Evaluation: Abstract Machine Models and Proxy Architectures for Exascale Computing
The strategic and economic impacts of high performance computing (HPC) systems, over the last several decades, have enabled dramatic improvements in manufacturing, design, development, and research, across almost every sector of the U.S. economy, including genetic analysis for agriculture and medicine, oil exploration, electronics, aircraft, and automotive design, and of course national defense technologies. HPC has repeatedly compounded reductions in time-to-solution and time-to-market in each of these areas. In recognition of these strategic and economic impacts, the Department of Energy (DOE) created the Exascale Computing Project (ECP) as a broad-based technology project focusing on simultaneously co-evolving high-performance computing architecture, system software, and application software. While many details of the exascale architectures are undefined, communicating the form and direction of this co-evolution of advances, in each of these three components, across the project, presents a need for a common language through which these advances can be shared. In this document we present a series of abstract representations designed to allow application developers to focus on the aspects of the machine that are important or relevant to performance and code structure. These abstract machine models (AMMs) describe the proposed architectures at the component and system level and are intended as communication aids between application developers and hardware architects during the co-design process. In addition to providing a common language for communication, the constraints of protecting the intellectual property of ECP’s vendor partners requires that the architectural advances be presented in an abstract sense, rather than exposing the specifics of an individual vendor’s designs, while still providing enough detail such that application developers will be able to reason about the performance trade-offs in the design space of their applications.
97 MATHEMATICS AND COMPUTING↗
High-Performance Key-Value Store on OpenSHMEM
Recently, there has been a growing interest in enabling fast data analytics by leveraging system capabilities from large-scale high-performance computing (HPC) systems. OpenSHMEM is a popular run-time system on HPC systems that has been used for large-scale compute-intensive scientific applications. In this paper, we propose to leverage OpenSHMEM to design a distributed in-memory key-value store for fast data analytics. Accordingly, we have developed SHMEMCache on top of OpenSHMEM to leverage its symmetric global memory, efficient one-sided communication operations and general portability. We have also evaluated SHMEMCache through extensive experimental studies. Our results show that SHMEMCache has accomplished significant performance improvements over hte original Memcached in terms of latency and throughput. Our evaluation on the Titan supercomputer has also demonstrated that SHMEMCache can scale to 1024 nodes.
97 MATHEMATICS AND COMPUTING↗
Estimating Lossy Compressibility of Scientific Data Using Deep Neural Networks
Simulation based scientific applications generate increasingly large amounts of data on high-performance computing (HPC) systems. To allow data to be stored and analyzed efficiently, data compression is often utilized to reduce the volume and velocity of data. However, a question often raised by domain scientists is the level of compression that can be expected so that they can make more informed decisions, balancing between accuracy and performance. In this letter, we propose a deep neural network based approach for estimating the compressibility of scientific data. To train the neural network, we build both general features as well as compressor-specific features so that the characteristics of both data and lossy compressors are captured in training. Our approach is demonstrated to outperform a prior analytical model as well as a sampling based approach in the case of a biased estimation, i.e., for SZ. However, for the unbiased estimation (i.e., ZFP), the sampling based approach yields the best accuracy, despite the high overhead involved in sampling the target dataset.
97 MATHEMATICS AND COMPUTING↗
Long Term Per-Component Power and Thermal Measurements of the OLCF Summit System
As we move into the exascale era, the power and energy footprints of high-performance computing (HPC) systems have grown significantly larger. Due to the harsh power and thermal conditions the system, components are exposed to extreme operating conditions. Operation of such modern HPC systems requires deep insights into long term system behavior to maintain its efficiency as well as its longevity. To help the HPC community to gain such insights, we provide a dataset that records the long-term power and thermal behavior of the 200PF pre-exascale supercomputer at the Oak Ridge Leadership Computing Facility (OLCF), Summit. This system is an IBM AC922 based system that has 9,252 IBM Power9 CPUs and 27,756 Nvidia V100 GPUs and can consume up to 13MW power at peak. Heat removal is performed using medium temperature direct liquid cooling and rear-door heat exchanger based secondary cooling loop. Originally extracted from a high-resolution (1Hz) per-component (GPUs, CPUs) measurements from the system, we primarily provide a dataset that has 10-second and 1-minute mean power and thermal measurements selected from five month-long segments over the course of 2020 (January and August), 2021 (February and August), and 2022 (January). For convenience, we also provide various sub datasets randomly sampled from the time and space (hosts) of the cluster. Further details and example code for analysis can be found in the following GitHub repository: https://github.com/at-aaims/summit_power_and_thermal_data
97 MATHEMATICS AND COMPUTING↗