Engineering topics

Jones, Terry

Publications and source records attributed to Jones, Terry.

At least 19 records

DOE OSTI2024

The ECP SICM project: Managing complex memory hierarchies for exascale applications

The Exascale Computing Project (ECP)’s Simplified Interface to Complex Memories (SICM) effort focuses on developing universal interfaces for discovering, managing, and sharing data across complex memory hierarchies. These facilitate the exploitation of emerging memory technologies and support precise control over their various trade-offs such as high-bandwidth versus low-latency, persistent versus ephemeral, high-capacity versus low-capacity, and near-CPU versus near-GPU. SICM comprises three interrelated components: a low-level interface, a high-level interface, and a persistent-heap interface. The low-level SICM interface is intended for system and run-time developers as well as expert application developers who prefer full control of the memory objects used within their application. The high-level SICM interface builds upon the low-level interface, employing application-level profiling and analysis to optimize data management for complex memory hierarchies. The persistent-heap interface provides applications with a persistent memory allocator that can allocate custom C++ data structures in both block-storage and byte-addressable persistent memories.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Autonomy Loops for Monitoring, Operational Data Analytics, Feedback, and Response in HPC Operations

Many High Performance Computing (HPC) facilities have developed and deployed frameworks in support of continuous monitoring and operational data analytics (MODA) to help improve efficiency and throughput. Because of the complexity and scale of systems and workflows and the need for low-latency response to address dynamic circumstances, automated feedback and response have the potential to be more effective than current human-in-the-loop approaches which are laborious and error prone. Progress has been limited, however, by factors such as the lack of infrastructure and feedback hooks, and successful deployment is often site- and case-specific. In this position paper we report on the outcomes and plans from a recent Dagstuhl Seminar, seeking to carve a path for community progress in the development of autonomous feedback loops for MODA, based on the established formalism of similar (MAPE-K) loops in autonomous computing and self-adaptive systems. By defining and developing such loops for significant cases experienced across HPC sites, we seek to extract commonalities and develop conventions that will facilitate interoperability and interchangeability with system hardware, software, and applications across different sites, and will motivate vendors and others to provide telemetry interfaces and feedback hooks to enable community development and pervasive deployment of MODA autonomy loops.

autonomy loops↗

DOE OSTI2023

Flexible and Effective Object Tiering for Heterogeneous Memory Systems

Computing platforms that package multiple types of memory, each with their own performance characteristics, are quickly becoming mainstream. To operate efficiently, heterogeneous memory architectures require new data management solutions that are able to match the needs of each application with an appropriate type of memory. As the primary generators of memory usage, applications create a great deal of information that can be useful for guiding memory tiering, but the community still lacks tools to collect, organize, and leverage this information effectively. To address this gap, this work introduces a novel software framework that collects and analyzes object-level information to guide memory tiering. Using this framework, this study evaluates and compares the impact of a variety of data tiering choices, including how the system prioritizes objects for faster memory as well as the frequency and timing of migration events. The results, collected on a modern Intel platform with conventional DRAM as well as non-volatile RAM, show that guiding data tiering with object-level information can enable significant performance and efficiency benefits compared to standard hardware- and software-directed data tiering strategies.

Kammerdiener, Brandon↗

DOE OSTI2022

Workshop Report: 2022 International Symposium of Quantitative Codesign of Supercomputers

This report provides general information about the Second International Symposium on the Quantitative Design of Supercomputers (SQCS). This new workshop was held in conjunction with Supercomputing ’22 on November 13, 2022, in Dallas Texas. The symposium aims at combining two methodologies—collaborative codesign and data-driven analysis—to realize the potential of supercomputing more fully. We refer to the design solutions that rely on intelligence from data-driven insights across applications, systems, system software, workflows, and facilities as quantitative codesign of supercomputers.

Jones, Terry↗

DOE OSTI2022

An Alternative Timing and Synchronization Approach for Situational Awareness and Predictive Analytics

Accurate and synchronized timing information is required by power system operators for controlling the grid infrastructure (relays, Phasor Measurement Units (PMUs), etc.) and determining asset positions. Satellite-based global positioning system (GPS) is the primary source of timing information. However, GPS disruptions today (both intentional and unintentional) can significantly compromise the reliability and security of our electric grids. A robust alternate source for accurate timing is critical to serve both as a deterrent against malicious attacks and as a redundant system in enhancing the resilience against extreme events that could disrupt the GPS network. To achieve this, we rely on the highly accurate, terrestrial atomic clock-based network for alternative timing and synchronization. In this paper, we discuss an experimental setup for an alternative timing approach. The data obtained from this experimental setup is continuously monitored and analyzed using various time deviation metrics. We also use these metrics to compute deviations of our clock with respect to the National Institute of Standards and Technologys (NIST) GPS data. The results obtained from these metric computations are elaborately discussed. Finally, we discuss the integration of the procedures involved, like real-time data ingestion, metric computation, and result visualization, in a novel microservices-based architecture for situational awareness.

Chinthavali, Supriya↗

DOE OSTI2022

Workshop Report: 2021 International Symposium of Quantitative Codesign of Supercomputers

This report provides general information about the First International Symposium on the Quantitative Design of Supercomputers (SQCS). This new workshop was held in conjunction with Supercomputing ’21 on November 19, 2021, in St Louis Missouri. The symposium aims at combining two methodologies—collaborative codesign and data-driven analysis—to realize the potential of supercomputing more fully. We refer to the design solutions that rely on intelligence from data-driven insights across applications, systems, system software, workflows, and facilities as quantitative codesign of supercomputers.

Jones, Terry↗

DOE OSTI2022

Position Papers for the ASCR Workshop on the Management and Storage of Scientific Data

The purpose of this workshop is to identify priority research directions in the area of data management for high-performance and scientific computing above and beyond HPC’s traditional "the parallel file system is the data-management system" model. Supporting the breadth of the DOE mission, including the explosion of AI uses and the growing needs of experimental and observational science, motivates revisiting our assumptions about data management. There are many facets of this topic to explore including: (1) Interfaces for accessing data that resides on traditional persistent storage as well as memory devices; (2) Storage-system architecture design that supports scientific workflows on varied hierarchical storage and networking devices; (3) Devising metadata management infrastructure to support FAIR principles (Findability, Accessibility, Interoperability, and Reusability); (4) Capturing provenance information about scientific data; (5) Utilizing AI to learn I/O patterns of emerging workloads for efficient data management; (6) Providing data management support for AI and complex workflows; and (7) Understanding the overlap between traditional storage systems and I/O (SSIO) efforts and data management. While the program committee has identified these topics as important areas for discussion, we welcome position papers from the community that propose additional topics of interest for discussion at the workshop. The workshop agenda will include breakout sessions for discussing these and selected topic areas to inform priority research directions for data management for high-performance and scientific computing.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Management and Storage of Scientific Data

Scientific discoveries rely heavily on efficient access, search, and management of massive data sets. Data management technologies have, for decades, provided foundational capabilities for scientific computing. Just as storage, input/output (I/O), and data management have been fundamental to simulation-based science for many years, so too are capable data-management technologies key to the success of today’s scientific workflows utilizing data intensive and machine learning (ML) techniques. The Department of Energy, Office of Science, Advanced Scientific Computing Research (ASCR) program has invested broadly in data-management research focused on high-performance computing (HPC) systems, from parallel file systems that store data to application software that makes these systems more productive. Still, advances in technology combined with growing diversity of supported science strongly motivate continued investment in this area. In January 2022, ASCR convened a workshop to identify priority research directions in the area of data management for high-performance and scientific computing. Attendees were challenged to identify promising approaches that would support the breadth of the DOE mission, including the explosion of artificial intelligence (AI) uses and the growing needs of experimental and observational science. Technological and science drivers were identified and considered as they relate to key aspects of data management such as interfaces, architectural design, and FAIR principles (Findable, Accessible, Interoperable, and Reusable). The thoughts of the workshop participants were distilled into a set of four priority research directions with the potential for high impact on DOE science. These research directions are summarized in the following pages.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2022

Report for the ASCR Workshop on the Management and Storage of Scientific Data

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Challenges to Updating Timing In The Power Grid.

Abstract not provided.

Jones, Terry↗

DOE OSTI2020

Performance Potential of Mixed Data Management Modes for Heterogeneous Memory Systems

Many high-performance systems now include different types of memory devices within the same compute platform to meet strict performance and cost constraints. Such heterogeneous memory systems often include an upper-level tier with better performance, but limited capacity, and lower-level tiers with higher capacity, but less bandwidth and longer latencies for reads and writes. To utilize the different memory layers efficiently, current systems rely on hardware-directed, memory -side caching or they provide facilities in the operating system (OS) that allow applications to make their own data-tier assignments. Since these data management options each come with their own set of trade-offs, many systems also include mixed data management configurations that allow applications to employ hardware- and software-directed management simultaneously, but for different portions of their address space. Despite the opportunity to address limitations of stand-alone data management options, such mixed management modes are under-utilized in practice, and have not been evaluated in prior studies of complex memory hardware. In this work, we develop custom program profiling, configurations, and policies to study the potential of mixed data management modes to outperform hardware- or software-based management schemes alone. Our experiments, conducted on an Intel ® Knights Landing platform with high-bandwidth memory, demonstrate that the mixed data management mode achieves the same or better performance than the best stand-alone option for five memory intensive benchmark applications (run separately and in isolation), resulting in an average speedup compared to the best stand-alone policy of over 10 %, on average.

Effler, Chad↗

DOE OSTI2019

Analyzing a Five-Year Failure Record of a Leadership-Class Supercomputer

Extreme-scale computing systems are required to solve some of the grand challenges in science and technology. From astrophysics to molecular biology, supercomputers are an essential tool to accelerate scientific discovery. However, large computing systems are prone to failures due to their complexity. It is crucial to develop an understanding of how these systems fail to design reliable supercomputing platforms for the future. This paper examines a five-year failure and workload record of a leadership-class supercomputer. To the best of our knowledge, five years represents the vast majority of the lifespan of a supercomputer. This is the first time such analysis is performed on a top 10 modern supercomputer. We performed a failure categorization and found out that: i) most errors are GPU-related, with roughly 37% of them being double-bit errors on the cards; ii) failures are not evenly spread across the physical machine, with room temperature presumably playing a major role; and iii) software errors of the system bring down several nodes concurrently. Our failure rate analysis unveils that: i) the system consistently degrades, being at least twice as reliable at the beginning, compared to the end of the period; ii) Weibull distribution closely fits the mean-time-between-failure data; and iii) hardware and software errors show a markedly different pattern. Finally, we correlated failure and workload records to reveal that: i) failure and workload records are weakly correlated, except for certain types of failures when segmented by the hours of the day; ii) several categories of failures make jobs crash within the first minutes of execution; and iii) a significant fraction of failed jobs exhaust the requested time with a disregard of when the failure occurred during execution.

Rojas, Elvis↗

DOE OSTI2019

Evaluating the effectiveness of program data features for guiding memory management

Recent trends have led to the adoption of larger and more complex memory systems, often with multiple tiers of memory performance within the same platform. To utilize complex memory systems efficiently, current data management strategies must be altered to map usage demands to the underlying hardware. Applications, as the generators of memory accesses, are well-suited to guide this process, but building and maintaining separate source code versions for different memory systems is not feasible in most cases. One potential solution is to employ automated program profiling and analysis to facilitate the production of application-based guidance. By attaching memory usage information to static or lightweight program features, compilers and runtime systems can generate fine-grained guidance without additional efforts from users or developers. Recent works have employed this approach with some success, but it is not clear which program features are most useful for guiding data management.This work evaluates the effectiveness of using different program data features to predict memory usage and guide memory management. It employs a custom set of simulation tools, based in the Intel® Pin framework, to collect and analyze the usage characteristics of application data associated with common program data features, such as allocation sites, types, and context. The results show that even relatively simple features, such as object size, are useful for selecting data with similar usage properties, but finer-grained features, such as the instructions that access a particular object, are often much more effective. Additionally, this work evaluates the performance of using different data features and different program inputs to guide data placement on a heterogeneous memory platform with a limited amount of high performance memory.

Effler, T. Chad↗

DOE OSTI2018

The EBEX Balloon-borne Experiment—Optics, Receiver, and Polarimetry

The E and B Experiment (EBEX) was a long-duration balloon-borne cosmic microwave background (CMB) polarimeter that flew over Antarctica in 2012. We describe the experiment's optical system, receiver, and polarimetric method and discuss their in-flight performance. EBEX had three frequency bands centered on 150, 250, and 410 GHz. To make effective use of limited mass and space, we created a 115 cm 2 sr high-throughput optical system that had two ambient temperature mirrors and four antireflection-coated polyethylene lenses per focal plane. All frequency bands shared the same optical train. Polarimetry was achieved with a continuously rotating achromatic half-wave plate (AHWP) that was levitated with a superconducting magnetic bearing (SMB). This is the first use of an SMB in astrophysics. Rotation stability was 0.45% over a period of 10 hr, and angular position accuracy was 0$^{°}_{.}$1. The measured modulation efficiency was above 90% for all bands. To our knowledge the 109% fractional bandwidth of the AHWP was the broadest implemented to date. The receiver, composed of one lens and the AHWP at a temperature of 4 K, the polarizing grid and other lenses at 1 K, and the two focal planes at 0.25 K, performed according to specifications, giving focal plane temperature stability with a fluctuation power spectrum that had a 1/f knee at 2 mHz. EBEX was the first balloon-borne instrument to implement technologies characteristic of modern CMB polarimeters, including high-throughput optical systems, and large arrays of transition edge sensor bolometric detectors with multiplexed readouts.

polarization↗

DOE OSTI2018

The EBEX Balloon-borne Experiment—Gondola, Attitude Control, and Control Software

The E and B Experiment (EBEX) was a long-duration balloon-borne instrument created to measure the polarization of the cosmic microwave background (CMB) radiation. EBEX was the first balloon-borne instrument to implement a kilopixel array of transition edge sensor (TES) bolometric detectors and the first CMB experiment to use the digital version of the frequency domain multiplexing system for readout of the TES array. The scan strategy relied on 40 s peak-to-peak constant-velocity azimuthal scans. We report on the unique demands on the design and operation of the payload that resulted from these new technologies and the scan strategy. We describe the solutions implemented, including the development of a power system designed to provide a total of at least 2.3 kW, a cooling system to dissipate 590 W consumed by the detectors' readout system, software to manage and handle the data of the kilopixel array, and specialized attitude reconstruction software. We demonstrate flight performance data reflecting faultless management of the TES array, adequate powering and cooling of the readout electronics, and constraint of attitude reconstruction errors such that the spurious B-modes they induced were less than 10% of the CMB B-mode power spectrum with r = 0.05.

polarization↗

DOE OSTI2018

The EBEX Balloon-borne Experiment—Detectors and Readout

EBEX was a long-duration balloon-borne experiment to measure the polarization of the cosmic microwave background. The experiment had three frequency bands centered at 150, 250, and 410 GHz and was the first to use a kilopixel array of transition edge sensor bolometers aboard a balloon platform. We determine the design and characterization of the array and the readout system. From the lowest to highest frequency, the median measured detectors' average thermal conductances were 39, 53, and 63 pW/K, the medians of transition temperatures were 0.45, 0.48, and 0.47 K, and the medians of normal resistances were 1.9, 1.5, and 1.4 Ω; moreover, we give the measured distributions. With the exception of the thermal conductance at 150 GHz, all measured values are within 30% of their design. We measure median low-loop-gain time constants τ 0 = 88, 46, and 57 ms. Two measurements of bolometer absorption efficiency gave results consistent within 10% and showing high (~0.9) efficiency at 150 GHz and medium (~0.35 and ~0.25) efficiency at the two higher bands. We calculate a median total optical power absorbed of 3.6, 5.3, and 5.0 pW. EBEX pioneered the use of the digital version of the frequency domain multiplexing system. We multiplexed the bias and readout of 16 bolometers onto two wires. The median per-detector noise-equivalent temperatures are 400, 920, and 14,500 $\mu {{\rm{K}}}_{{\rm{cmb}}}\sqrt{{\rm{s}}}$. We compare these values to our preflight predictions and to a previous balloon payload. We report on the sources of excess noise and the path for a future payload to make full use of the balloon environment.

instrumentation: polarimeters↗

DOE OSTI2018

A Program Analysis Tool to Query Static and Dynamic Characteristics of HPC Applications

Emerging HPC platforms are becoming extraordinarily difficult to program as a result of complex, deep and heterogeneous memory hierarchies, heterogeneous cores, and the need to divide work and data among them. New programming models and libraries are being developed to aid in porting efforts of application developers, but substantial code restructuring is still necessary to fully make use of these new technologies. To do this effectively, these developers need information about their source code characteristics, including static and dynamic (e.g. performance) information to direct their optimisation efforts and make key decisions. On the other hand, system administrators need to understand how users are using the software stack and resources on their systems to understand what software they need to provide to improve the productivity of the users on a platform.In this paper we describe a tool that combines compiler and profiler information to query performance characteristics within an application or across multiple applications on a given platform. Static and dynamic data about applications are collected and stored together in an SQL database that can be queried by either a developer or a system administrator. We will demonstrate the capabilities of this tool via a real-world example from application-driven case studies that aims at understanding the use of scientific libraries on a routine from the molecular simulation application CP2K.

Welch, Aaron↗

DOE OSTI2017

An evaluation of the state of time synchronization on leadership class supercomputers

We present a detailed examination of time agreement characteristics for nodes within extreme-scale parallel computers. Using a software tool we introduce in this paper, we quantify attributes of clock skew among nodes in three representative high-performance computers sited at three national laboratories. Our measurements detail the statistical properties of time agreement among nodes and how time agreement drifts over typical application execution durations. We discuss the implications of our measurements, why the current state of the field is inadequate, and propose strategies to address observed shortcomings.

97 MATHEMATICS AND COMPUTING↗