Seamless Communication Between High-Performance Computing System and Electron Microscopes for On-Demand Automated Data Transfer and Remote Control
Explore the source record for details and available documents.
Mukherjee, Debangshu↗
SEARCH · Search NASA
Results for “computer system performance”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Exploiting communication concurrency on high performance computing systems
Although logically available, applications may not exploit enough instantaneous communication concurrency to maximize hardware utilization on HPC systems. This is exacerbated in hybrid programming models such as SPMD+OpenMP. We present the design of a "multi-threaded" runtime able to transparently increase the instantaneous network concurrency and to provide near saturation bandwidth, independent of the application configuration and dynamic behavior. The runtime forwards communication requests from application level tasks to multiple communication servers. Our techniques alleviate the need for spatial and temporal application level message concurrency optimizations. Experimental results show improved message throughput and bandwidth by as much as 150% for 4KB bytes messages on InfiniBand and by as much as 120% for 4KB byte messages on Cray Aries. For more complex operations such as all-to-all collectives, we observe as much as 30% speedup. This translates into 23% speedup on 12,288 cores for a NAS FT implemented using FFTW. We also observe as much as 76% speedup on 1,500 cores for an already optimized UPC+OpenMP geometric multigrid application using hybrid parallelism.
97 MATHEMATICS AND COMPUTING↗
Scalable Energy Efficiency with Resilience for High Performance Computing Systems: A Quantitative Methodology
Energy efficiency and resilience are two crucial challenges for HPC systems to reach exascale. While energy efficiency and resilience issues have been extensively studied individually, little has been done to understand the interplay between energy efficiency and resilience for HPC systems. Decreasing the supply voltage associated with a given operating frequency for processors and other CMOS-based components can significantly reduce power consumption. However, this often raises system failure rates and consequently increases application execution time. In this work, we present an energy saving undervolting approach that leverages the mainstream resilience techniques to tolerate the increased failures caused by undervolting.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
HPGMG 1.0: A Benchmark for Ranking High Performance Computing Systems
This document provides an overview of the benchmark - HPGMG - for ranking large scale general purpose computers for use on the Top500 list. We provide a rationale for the need for a replacement for the current metric HPL, some background of the Top500 list and the challenges of developing such a metric; we discuss our design philosophy and methodology, and an overview of the specification of the benchmark. The primary documentation with maintained details on the specification can be found at hpgmg.org and the Wiki and benchmark code itself can be found in the repository https://bitbucket.org/hpgmg/hpgmg.
97 MATHEMATICS AND COMPUTING↗
Scalable Energy Efficiency with Resilience for High Performance Computing Systems: A Quantitative Methodology
Energy efficiency and resilience are two crucial challenges for HPC systems to reach exascale. While energy efficiency and resilience issues have been extensively studied individually, little has been done to understand the interplay between energy efficiency and resilience for HPC systems. Decreasing the supply voltage associated with a given operating frequency for processors and other CMOS-based components can significantly reduce power consumption. However, this often raises system failure rates and consequently increases application execution time. In this work, we present an energy saving undervolting approach that leverages the mainstream resilience techniques to tolerate the increased failures caused by undervolting.
energy, resilience↗
Thermal, Power, and Co-location Aware Resource Allocation in Heterogeneous High Performance Computing Systems
Explore the source record for details and available documents.
Oxley, Mark A.↗
High-Performance Computing for Earth System Modeling
High-performance computing (HPC) plays an important role during the development of Earth system models. This chapter reviews HPC efforts related to Earth system models, including community Earth system models and energy exascale Earth system models. Specifically, this chapter evaluates computational and software design issues, analyzes several current HPC-related model developments, and provides an outlook for some promising areas within Earth system modeling in the era of exascale computing.
Wang, Dali↗
Enforcing End-to-end I/O Policies for Scientific Workflows using Software-Defined Storage Resource Enclaves
Data-intensive knowledge discovery requires scientific applications to run concurrently with analytics and visualization codes executing in situ for timely output inspection and knowledge extraction. Consequently, I/O pipelines of scientific workflows can be long and complex because they comprise many stages of analytics across different layers of the I/O stack of high-performance computing systems. Performance limitations at any I/O layer or stage can cause an I/O bottleneck resulting in greater than expected end-to-end I/O latency. In this paper, we present the design and implementation of a novel data management infrastructure called Software-Defined Storage Resource Enclaves (SIREN) at system level to enforce end-to-end policies that dictate an I/O pipeline's performance. SIREN provides an I/O performance interface for users to specify the desired storage resources in the context of in-situ analytics. If suboptimal performance of analytics is caused by an I/O bottleneck when data are transferred between simulations and analytics, schedulers in different layers of the I/O stack automatically provide the guaranteed lower bounds on I/O throughput. Lastly, our experimental results demonstrate that SIREN provides performance isolation among scientific workflows sharing multiple storage servers across two I/O layers (burst buffer and parallel file systems) while maintaining high system scalability and resource utilization.
97 MATHEMATICS AND COMPUTING↗
Special Issue on Data-Intensive Scalable Computing Systems
Traditional high performance computing (HPC) systems were designed from a compute-centric perspective, with an emphasis on high floating point performance. As scientific and analytics applications become more data intensive, there is a need to rethink HPC system architectures, programming models, runtime systems, and tools with a focus on scalable data intensive computing. Industry approaches supporting data intensive applications have been highly successful, leading many in the HPC community to search for ways to adapt them to HPC hardware and software contexts. Conversely, industry looks to the HPC community for their expertise in designing, deploying, and using extremely large and high performance computing systems. This special issue contains seven articles, each of which is an extended version of a paper presented in the 2015 International Workshop on Data Intensive Scalable Computing Systems (DISCS-2015) held in Austin, Texas on November 15, 2015. The workshop program included papers that ranged from programming models to systems software to storage, each with a strong focus on data intensive computing.
97 MATHEMATICS AND COMPUTING↗
A survey on resource allocation in high performance distributed computing systems
An efficient resource allocation is a fundamental requirement in high performance computing (HPC) systems. Many projects are dedicated to large-scale distributed computing systems that have designed and developed resource allocation mechanisms with a variety of architectures and services. In our study, through analysis, a comprehensive survey for describing resource allocation in various HPCs is reported. The aim of the work is to aggregate under a joint framework, the existing solutions for HPC to provide a thorough analysis and characteristics of the resource management and allocation strategies. Resource allocation mechanisms and strategies play a vital role towards the performance improvement of all the HPCs classifications. Therefore, a comprehensive discussion of widely used resource allocation strategies deployed in HPC environment is required, which is one of the motivations of this survey. Moreover, we have classified the HPC systems into three broad categories, namely: (a) cluster, (b) grid, and (c) cloud systems and define the characteristics of each class by extracting sets of common attributes. All of the aforementioned systems are cataloged into pure software and hybrid/hardware solutions. The system classification is used to identify approaches followed by the implementation of existing resource allocation strategies that are widely presented in the literature.
97 MATHEMATICS AND COMPUTING↗
Utility maximizing dynamic resource management in an oversubscribed energy-constrained heterogeneous computing system
The need for greater performance in high performance computing systems combined with rising costs of electricity to power these systems motivates the need for energy-efficient resource management. Driven by the requirements of the Extreme Scale Systems Center at Oak Ridge National Laboratory, our study addresses the problem of scheduling dynamically-arriving tasks to machines in an oversubscribed and energy-constrained heterogeneous distributed computing environment. Our goal is to maximize total “utility” earned by the system, where the utility of a task is defined by a monotonically-decreasing function that represents the value of completing that task at different times. To address this problem, we design four energy-aware resource allocation heuristics and compare their performance to heuristics from the literature. For our given energy-constrained environment, we also design an energy filtering technique that helps some heuristics regulate their energy consumption by allowing tasks to only consume up to an estimated fair-share of energy. Extensive sensitivity analyses of the heuristics in environments with different levels of heterogeneity show that heuristics with the ability to balance both energy consumption and utility exhibit the best performance because they save energy for use by future tasks.
97 MATHEMATICS AND COMPUTING↗
Software Systems for High-performance Quantum Computing
Quantum computing promises new opportunities for solving hard computational problems, but harnessing this novelty requires breakthrough concepts in the design, operation, and application of computing systems. We define some of the challenges facing the development of quantum computing systems as well as software-based approaches that can be used to overcome these challenges. Following a brief overview of the state of the art, we present models for the quantum programming and execution models, the development of architectures for hybrid high-performance computing systems, and the realization of software stacks for quantum networking. This leads to a discussion of the role that conventional computing plays in the quantum paradigm and how some of the current challenges for exascale computing overlap with those facing quantum computing.
high-performance computing↗
Methods, apparatus and system for selective duplication of subtasks
A method for selective duplication of subtasks in a high-performance computing system includes: monitoring a health status of one or more nodes in a high-performance computing system, where one or more subtasks of a parallel task execute on the one or more nodes; identifying one or more nodes as having a likelihood of failure which exceeds a first prescribed threshold; selectively duplicating the one or more subtasks that execute on the one or more nodes having a likelihood of failure which exceeds the first prescribed threshold; and notifying a messaging library that one or more subtasks were duplicated.
97 MATHEMATICS AND COMPUTING↗
Methods, apparatus and system for selective duplication of subtasks
A method for selective duplication of subtasks in a high-performance computing system includes: monitoring a health status of one or more nodes in a high-performance computing system, where one or more subtasks of a parallel task execute on the one or more nodes; identifying one or more nodes as having a likelihood of failure which exceeds a first prescribed threshold; selectively duplicating the one or more subtasks that execute on the one or more nodes having a likelihood of failure which exceeds the first prescribed threshold; and notifying a messaging library that one or more subtasks were duplicated.
Andrade Costa, Carlos H.↗
ChemComp: Compiling and Computing with Chemical Reaction Networks
The exponential growth in computing demands driven by scientific computing, data analytics, and artificial intelligence is pushing conventional CMOS-based high-performance computing systems to their physical and energy efficiency limits. As we approach the era of post-exascale computing, disruptive approaches are necessary to overcome these barriers and achieve substantial gains in energy efficiency. Analog and hybrid digital-analog computing systems have emerged as promising alternatives, offering the potential for orders-of-magnitude improvements in efficiency. Among these, biochemical computing stands out as a novel paradigm capable of leveraging the natural efficiency of chemical reactions, which have shown promise in solving optimization problems by converging to steady states. By scaling up reaction networks or reaction vessel sizes, biochemical systems present an opportunity to meet the high-performance demands of modern computing tasks. Despite their promise, significant theoretical and practical challenges remain, particularly in formulating and mapping computational problems to chemical reaction networks (CRNs) and designing viable biochemical computing devices. This paper addresses these challenges by introducing new ideas to ChemComp, a compilation and emulation framework for chemical computation. This work describes the mechanisms through which solutions to ordinary differential equations (ODEs) that can be represented as CRN systems can be achieved. Furthermore, we explain the design principles of an ODE dialect implemented as a multi-level intermediate representation (MLIR) compiler extension that will be coupled with existing infrastructure. We demonstrate the potential of our framework through a case study emulating a simplified chemical reservoir computing device. This work establishes foundational tools and methodologies necessary to harness the computational power of chemistry, paving the way for the development of energy-efficient, high-performance computing systems tailored to contemporary and future computational needs.
Bohm Agostini, Nicolas↗
Opportunities for nonvolatile memory systems in extreme-scale high-performance computing
For extreme-scale high-performance computing systems, system-wide power consumption has been identified as one of the key constraints moving forward, where DRAM main memory systems account for about 30 to 50 percent of a node's overall power consumption. As the benefits of device scaling for DRAM memory slow, it will become increasingly difficult to keep memory capacities balanced with increasing computational rates offered by next-generation processors. However, several emerging memory technologies related to nonvolatile memory (NVM) devices are being investigated as an alternative for DRAM. Moving forward, NVM devices could offer solutions for HPC architectures. Researchers are investigating how to integrate these emerging technologies into future extreme-scale HPC systems and how to expose these capabilities in the software stack and applications. In addition, current results show several of these strategies could offer high-bandwidth I/O, larger main memory capacities, persistent data structures, and new approaches for application resilience and output postprocessing, such as transaction-based incremental checkpointing and in situ visualization, respectively.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
FY 2021 High Performance Computing Overall Systems Availability
INL HPC capabilities were utilized by individuals at universities, industry, and government laboratories for a wide range of research and engineering activities between October 1, 2020 and September 30, 2021. The HPC systems themselves experienced minimal downtime over the course of the fiscal year. Most downtime was planned to apply critical security patches to system software. The most serious of the unplanned downtime events occurred due to maintenance on the Kohler backup generator in July. Users were kept notified of all events, both planned and unplanned, via 36 total email notifications in FY-21. The INL HPC systems continue to provide high availability for collaboration and innovation in nuclear energy systems research.
99 GENERAL AND MISCELLANEOUS↗
Memory Errors in Modern Systems: The Good, The Bad, and The Ugly
Several recent publications have shown that hardware faults in the memory subsystem are commonplace. These faults are predicted to become more frequent in future systems that contain orders of magnitude more DRAM and SRAM than found in current memory subsystems. These memory subsystems will need to provide resilience techniques to tolerate these faults when deployed in high-performance computing systems and data centers containing tens of thousands of nodes. Therefore, it is critical to understand the efficacy of current hardware resilience techniques to determine whether they will be suitable for future systems. In this paper, we present a study of DRAM and SRAM faults and errors from the field. We use data from two leadership-class high-performance computer systems to analyze the reliability impact of hardware resilience schemes that are deployed in current systems. Our study has several key findings about the efficacy of many currently deployed reliability techniques such as DRAM ECC, DDR address/command parity, and SRAM ECC and parity. We also perform a methodological study, and find that counting errors instead of faults, a common practice among researchers and data center operators, can lead to incorrect conclusions about system reliability. Lastly, we use our data to project the needs of future large-scale systems. We find that SRAM faults are unlikely to pose a significantly larger reliability threat in the future, while DRAM faults will be a major concern and stronger DRAM resilience schemes will be needed to maintain acceptable failure rates similar to those found on today's systems.
97 MATHEMATICS AND COMPUTING↗