Durango: Scalable Synthetic Workloard Generation for Extreme-Scale Application Performance Modeling & Simulation
Explore the source record for details and available documents.
Carothers, Christopher D.↗
SEARCH · Search NASA
Results for “Extreme-scale computing”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Parked aeroelastic field rotor response for a 20% scaled demonstrator of a 13‐MW downwind turbine
Abstract Aeroelastic parked testing of a unique downwind two‐bladed subscale rotor was completed to characterize the response of an extreme‐scale 13‐MW turbine in high‐wind parked conditions. A 20% geometric scaling was used resulting in scaled 20‐m‐long blades, whose structural and stiffness properties were designed using aeroelastic scaling to replicate the nondimensional structural aeroelastic deflections and dynamics that would occur for a lightweight, downwind 13‐MW rotor. The subscale rotor was mounted and field tested on the two‐bladed Controls Advanced Research Turbine (CART2) at the National Renewable Energy Laboratory's Flatiron Campus (NREL FC). The parked testing of these highly flexible blades included both pitch‐to‐run and pitch‐to‐feather configurations with the blades in the horizontal braked orientation. The collected experimental data includes the unsteady flapwise root bending moments and tip deflections as a function of inflow wind conditions. The bending moments are based on strain gauges located in the root section, whereas the tip deflections are captured by a video camera on the hub of the turbine pointed toward the tip of the blade. The experimental results are compared against computational predictions generated by FAST, a wind turbine simulation software, for the subscale and full‐scale models with consistent unsteady wind fields. FAST reasonably predicted the bending moments and deflections of the experimental data in terms of both the mean and standard deviations. These results demonstrate the efficacy of the first such aeroelastically scaled turbine test and demonstrate that a highly flexible lightweight downwind coned rotor can be designed to withstand extreme loads in parked conditions.
17 WIND ENERGY↗
Constructing a new predictive scaling formula for ITER's divertor heat-load width informed by a simulation-anchored machine learning
Understanding and predicting divertor heat-load width λq is a critically important problem for an easier and more robust operation of ITER with high fusion gain. Previous predictive simulation data for λ q using the extreme-scale edge gyrokinetic code XGC1 [S. Ku et al., Phys. Plasmas 25, 056107 (2018)] in the electrostatic limit under attached divertor plasma conditions in three major US tokamaks [C. S. Chang et al., Nucl. Fusion 57, 116023 (2017)] reproduced the Eich and Goldston attached-divertor formula results [formula #14 in T. Eich et al., Nucl. Fusion 53, 093031 (2013) and R. J. Goldston, Nucl. Fusion 52, 013009 (2012)] and furthermore predicted over six times wider λ q than the maximal Eich and Goldston formula predictions on a full-power (Q = 10) scenario ITER plasma. After adding data from further predictive simulations on a highest current JET and highest-current Alcator C-Mod, a machine learning program is used to identify a new scaling formula for λ q as a simple modification to the Eich formula #14, which reproduces the Eich scaling formula for the present tokamaks and which embraces the wide λ q XGC for the full-current Q = 10 ITER plasma. Additionally, the new formula is then successfully tested on three more ITER plasmas: two corresponding to long burning scenarios with Q = 5 and one at low plasma current to be explored in the initial phases of ITER operation. The new physics that gives rise to the wider λ q XGC is identified to be the weakly collisional, trapped-electron-mode turbulence across the magnetic separatrix, which is known to be an efficient transporter of the electron heat and mass. Electromagnetic turbulence and high-collisionality effects on the new formula are the next study topics for XGC1.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Benchmark Generation and Simulation at Extreme Scale
The path to extreme scale high-performance computing (HPC) poses several challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Investigating the performance of parallel applications at scale on future architectures and the performance impact of different architectural choices is an important component of HPC hardware/software co-design. Simulations using models of future HPC systems and communication traces from applications running on existing HPC systems can offer an insight into the performance of future architectures. This work targets technology developed for scalable application tracing of communication events. It focuses on extreme-scale simulation of HPC applications and their communication behavior via lightweight parallel discrete event simulation for performance estimation and evaluation. Instead of simply replaying a trace within a simulator, this work promotes the generation of a benchmark from traces. This benchmark is subsequently exposed to simulation using models to reflect the performance characteristics of future-generation HPC systems. This technique provides a number of benefits, such as eliminating the data intensive trace replay and enabling simulations at different scales. The presented work features novel software co-design aspects, combining the ScalaTrace tool to generate scalable trace files, the ScalaBenchGen tool to generate the benchmark, and the xSim tool to assess the benchmark characteristics within a simulator.
Lagadapati, Mahesh↗
Running Ensemble Workflows at Extreme Scale: Lessons Learned and Path Forward
The ever-increasing volumes of scientific data combined with sophisticated techniques for extracting information from them have led to the increasing popularity of ensemble workflows which are a collection of runs of individual workflows. A traditional approach followed by scientists to run ensembles is to rely on simple scripts to execute different runs and manage resources. This approach is not scalable and is error-prone, thereby motivating the development of workflow management systems that specialize in executing ensembles on HPC clusters. However, when the size of both the ensemble and the target system reach extreme scales, existing workflow management systems face new challenges that hamper their efficient execution. In this paper, we describe our experience scaling an ensemble workflow from the computational biology domain from the early design stages to the execution at extreme scale on Summit, a leadership class supercomputer at the Oak Ridge National Laboratory. We discuss challenges that arise when scaling ensembles to several million runs on thousands of HPC nodes. We identify challenges with composition of the ensemble itself, its execution at large scale, post-processing of the generated data, and scalability of the file system. Based on the experience acquired, we develop a generic vision of the capabilities and abstractions to add to existing workflow management systems to enable the execution of ensemble workflows at extreme scales. We believe that the understanding of these fundamental challenges will help application teams along with workflow system developers with designing the next generation of infrastructure for composing and executing extreme-scale ensemble workflows.
Mehta, Kshitij↗
Exploring Properties and Correlations of Fatal Events in a Large-Scale HPC System
In this study, we explore potential correlations of fatal system events for one of the most powerful supercomputers-IBM Blue Gene/Q Mira, which is deployed at Argonne National Laboratory, based on its 5-year reliability, availability, and serviceability (RAS) log. Our contribution is two-fold. (1) We design an efficient log analysis tool, namely LogAider, with a novel filtering method to effectively extract fatal events from masses of system messages that are heavily duplicated in the log. LogAider exhibits a very precise detection of temporal-correlation with a high similarity (up to 95 percent) to the ground-truth (i.e., compared to the failure records reported by the administrators). The total number of fatal events can be reduced to about 1,255 compared with originally 2.6 million duplicated fatal messages. (2) We analyze the 5-year RAS log of the MIRA system using LogAider, and summarize six important "takeaways" which can help system vendors and administrators better understand an extreme-scale system's fatal events. Specifically, we find that the distribution or proportion of the fatal system events follow a Pareto-like principle in general. The temporal correlation among fatal events is much stronger than that of warn messages and info messages, and the correlated events tend to constitute a few clusters. The mean time between fatal events (MTBFE) of the Mira system is about 1.3 days from the perspective of the system, and the MTTI is 2-4 days from the perspective of users. The most error-prone item value with respect to any key attribute appears likely in the log every 2-10 days. Weibull, Gamma, and Pearson6 are the three best-fit distributions for the fatal event intervals. The overall correlation of fatal events on the 5D torus network is not prominent, whereas the small-region locality correlation (e.g., the fatal events inside racks) is relatively strong. We believe our work will be interesting to large-scale HPC system administrators and vendors and to fault tolerance researchers, enabling them to better understand fatal events and mitigate such events accordingly.
97 MATHEMATICS AND COMPUTING↗
A Pattern Language for High-Performance Computing Resilience
High-performance computing systems (HPC) provide powerful capabilities for modeling, simulation, and data analytics for a broad class of computational problems. They enable extreme performance of the order of quadrillion floating-point arithmetic calculations per second by aggregating the power of millions of compute, memory, networking and storage components. With the rapidly growing scale and complexity of HPC systems for achieving even greater performance, ensuring their reliable operation in the face of system degradations and failures is a critical challenge. System fault events often lead the scientific applications to produce incorrect results, or may even cause their untimely termination. The sheer number of components in modern extreme-scale HPC systems and the complex interactions and dependencies among the hardware and software components, the applications, and the physical environment makes the design of practical solutions that support fault resilience a complex undertaking. To manage this complexity, we developed a methodology for designing HPC resilience solutions using design patterns. We codified the well-known techniques for handling faults, errors and failures that have been devised, applied and improved upon over the past three decades in the form of design patterns. In this paper, we present a pattern language to enable a structured approach to the development of HPC resilience solutions. The pattern language reveals the relations among the resilience patterns and provides the means to explore alternative techniques for handling a specific fault model that may have different efficiency and complexity characteristics. Using the pattern language enables the design and implementation of comprehensive resilience solutions as a set of interconnected resilience patterns that can be instantiated across layers of the system stack.
Hukerikar, Saurabh↗
Fit Fly: A Case Study of Interconnect Innovation through Parallel Simulation
To meet the demand for exascale-level performance from high-performance computing (HPC) interconnects, many system architects are turning to simulation results for accurate and reliable predictions of the performance of prospective technologies. Testing full-scale networks with a variety of benchmarking tools, including synthetic workloads and application traces, can give crucial insight into what ideas are most promising without needing to physically construct a test network. While flexible, however, this approach is extremely compute time intensive. We address this time complexity challenge through the use of large-scale, optimistic parallel simulation that ultimately leads to faster HPC network architecture innovations. In this paper we demonstrate this innovation capability through a real-world network design case study. Specifically, we have simulated and compared four extreme-scale interconnects: Dragonfly, Megafly, Slim Fly, and a new dual-rail-dual-plane variation of the Slim Fly network topology. We present this new variant of Slim Fly, dubbed Fit Fly, to show how interconnect innovation and evaluation-beyond what is possible through analytic methods-can be achieved through parallel simulation. We validate and compare the model with various network designs using the CODES interconnect simulation framework. By running large-scale simulations in a parallel environment, we are able to quickly generate reliable performance results that can help network designers break ground on the next generation of high-performance network designs.
High Performance Computing↗
Quantum Dynamics at Scale: Ultrafast Control of Emergent Functional Materials
Confluence of extreme-scale quantum dynamics simulations (i.e. quantum@scale) and cutting-edge x-ray free-electron laser experiments are revolutionizing materials science. An archetypal example is the exciting concept of using picosecond light pulses to control emergent material properties on demand in atomically-thin layered materials. This paper describes efforts to scale our quantum molecular dynamics engine toward the United States' first exaflop/s computer, under an Aurora Early Science Program project named "Metascalable layered material genome". Key algorithmic and computing techniques incorporated are: (1) globally-scalable and locally-fast solvers within a linear-scaling divide-conquer-recombine algorithmic framework; (2) algebraic `BLASification' of computational kernels; and (3) data alignment and loop restructuring, along with register and cache blocking, for enhanced vectorization and efficient memory access. The resulting weak-scaling parallel efficiency was 0.93 on 131,072 Intel Xeon Phi cores for a 56.6 million atom (or 169 million valence-electron) system, whereas the various code transformations achieved 5-fold speedup. The optimized simulation engine allowed us for the first time to establish a significant effect of substrate on the dynamics of layered material upon electronic excitation.
Algebraic BLASification↗
Programmer-guided reliability for extreme-scale applications
We present “programmer-guided reliability” (PGR) as a systematic conceptual approach to address the expected rise in soft errors in coming extreme-scale systems at the application level. The approach involves instrumentation of the application with code to detect data corruption errors. The location and nature of these error detectors are at the discretion of the programmer, who uses their knowledge and experience with the problem domain, the application, the solution algorithms, etc., to determine the most vulnerable areas of the code and the most appropriate ways to detect data corruption. To illustrate the approach, we provide examples of error detectors from four different benchmark-scale applications. We also describe a simple control framework that allows for runtime configuration of the error detectors without recompilation of the application, as well as dynamic reconfiguration during the execution of the application. Lastly, we discuss a number of future directions building on the basic PGR approach, including the incorporation of some general error detectors into the programming environment in order to make them more easily usable by the programmer.
97 MATHEMATICS AND COMPUTING↗
Highlights of X-Stack ExM Deliverable Swift/T
Swift/T is a key success from the ExM: System support for extreme-scale, many-task applications1 X-Stack project, which proposed to use concurrent dataflow as an innovative programming model to exploit extreme parallelism in exascale computers. The Swift/T component of the project reimplemented the Swift language from scratch to allow applications that compose scientific modules together to be build and run on available petascale computers (Blue Gene, Cray). Swift/T does this via a new compiler and runtime that generates and executes the application as an MPI program. We assume that mission-critical emerging exascale applications will be composed as scalable applications using existing software components, connected by data dependencies. Developers wrap native code fragments using a higherlevel language, then build composite applications to form a computational experiment. This exemplifies hierarchical concurrency: lower-level messaging libraries are used for fine-grained parallelism; highlevel control is used for inter-task coordination. These patterns are best expressed with dataflow, but static DAGs (i.e., other workflow languages) limit the applications that can be built; they do not provide the expressiveness of Swift, such as conditional execution, iteration, and recursive functions.
97 MATHEMATICS AND COMPUTING↗
PIPER: Performance Insight for Programmers and Exascale Runtimes: Guiding the Development of the Exascale Software Stack
The PIPER project set out to develop methodologies and software for measurement, analysis, attribution, and presentation of performance data for extreme-scale systems. Goals of the project were to support analysis of massive multi-scale parallelism, heterogeneous architectures, multi-faceted performance concerns, and to support both post-mortem performance analysis to identify program features that contribute to problematic performance and on-line performance analysis to drive adaptation. This final report summarizes the research and development activity at Rice University as part of the PIPER project. Producing a complete suite of performance tools for exascale platforms during the course of this project was impossible since both hardware and software for exascale systems is still a moving target. For that reason, the project focused broadly on the development of new techniques for measurement and analysis of performance on modern parallel architectures, enhancements to HPCToolkit’s software infrastructure to support our research goals or use on sophisticated applications, engaging developers of multithreaded runtimes to explore how support for tools should be integrated into their designs, engaging operating system developers with feature requests for enhanced monitoring support, engaging vendors with requests that they add hardware measure- ment capabilities and software interfaces needed by tools as they design new components of HPC platforms including processors, accelerators and networks, and finally collaborations with partners interested in using HPCToolkit to analyze and tune scalable parallel applications.
97 MATHEMATICS AND COMPUTING↗
Scalable Multilevel Uncertainty Quantification Concepts for Extreme-Scale Multiscale Problems
Our work in this project is aimed at making fundamental advances in multilevel and multiscale methods for problems with uncertainties related to flows in porous media. The main thrust of this research is to develop a systematic multiscale, multilevel parallel approaches for problems with uncertainties and use them in the uncertainty quantification. A key emphasis is on problems without an apparent scale separation in space, time, and uncertainty spaces. Multiscale solution methods are currently under active investigation. However, the develop- ment of multiscale methods for problems with uncertainties where the scales and uncertainties are tightly coupled is a research area that is less explored. During this project, we have made some significant contributions to the developments of multiscale methods, the uncertainty quantification for multiscale methods, multiscale model learning, uncertainty quantification in inverse problems, and applications.
97 MATHEMATICS AND COMPUTING↗
Optimizing the Energy Usage and Cognitive Value of Extreme-Scale Data-Analysis Approaches
This report serves as a record that we have successfully completed work on the proposal Optimizing the Energy Usage and Cognitive Value of Extreme Scale Analysis Approaches, or, more succinctly, ECX. This includes successful completion of all three phases of the project, findings as summarized below, strong publications across the fields represented in the proposal, and a set of artifacts (code, websites, and other documentation) available to the public. In total, the results of this project represent significant findings in each domain, and a toolkit that will allow the community to engage with this work, advancing it and contributing to it in the years ahead. They also form the basis for a successfully funded follow-on project. This executive summary presents findings, tools and publications resulting from the ECX project. The balance of the report then provides more detail on these results.
97 MATHEMATICS AND COMPUTING↗
Optimizing the Energy Usage and Cognitive Value of Extreme-Scale Data-Analysis Approaches
This report serves as a record that we have successfully completed work on the proposal Optimizing the Energy Usage and Cognitive Value of Extreme Scale Analysis Approaches, or, more succinctly, ECX. This includes successful completion of all three phases of the project, findings as summarized below, strong publications across the fields represented in the proposal, and a set of artifacts (code, websites, and other documentation) available to the public. In total, the results of this project represent significant findings in each domain, and a toolkit that will allow the community to engage with this work, advancing it and contributing to it in the years ahead. They also form the basis for a successfully funded follow-on project.
97 MATHEMATICS AND COMPUTING↗
Optimizing the Energy Usage and Cognitive Value of Extreme-Scale Data-Analysis Approaches (Final Report)
This report serves as a record that we have successfully completed work on the proposal Optimizing the Energy Usage and Cognitive Value of Extreme Scale Analysis Approaches, or, more succinctly, ECX. This includes successful completion of all three phases of the project, findings as summarized below, strong publications across the fields represented in the proposal, and a set of artifacts (code, websites, and other documentation) available to the public. In total, the results of this project represent significant findings in each domain, and a toolkit that will allow the community to engage with this work, advancing it and contributing to it in the years ahead. They also form the basis for a successfully funded follow-on project. This executive summary presents findings, tools and publications resulting from the ECX project. The balance of the report then provides more detail on these results.
97 MATHEMATICS AND COMPUTING↗
Program Verification for Extreme-Scale Applications (Final Scientific/Technical Report)
The project seeks to develop tools and techniques to help software developers verify the correctness and accuracy of their code. The target domain is scientific software of the kind widely used and developed in the Department of Energy research community, with a particular focus on "extreme scale" programs - those that are expected to involve possibly millions of parallel threads of execution. The report covers the University of Delaware contribution to the collaborative project. The project had a number of successful outcomes, especially regarding the development and extension of the CIVL software verification framework. CIVL is a verification tool for C or Fortran programs that use MPI, OpenMP, CUDA, and/or Pthreads for parallelization. CIVL went through vast improvements and extensions, and was successfully applied to a number of challenging codes. It found a subtle bug in the Devito PDE framework. It was able to verify the functional correctness of a conjugate gradient solver using a novel probabilistic technique with vanishingly small chance of error. CIVL was used very successfully in verification competitions, and the CIVL solutions to the competition challenges were presented and published. CIVL was also used successfully by other researchers on a computational chemistry kernel.
97 MATHEMATICS AND COMPUTING↗
Enabling Structured Exploration of Workflow Performance Variability in Extreme-Scale Environments
Workflows are taking an Workflows are taking an increasingly important role in orchestrating complex scientific processes in extreme scale and highly heterogeneous environments. However, to date we cannot reliably predict, understand, and optimize workflow performance. Sources of performance variability and in particular the interdependencies of workflow design, execution environment and system architecture are not well understood. While there is a rich portfolio of tools for performance analysis, modeling and prediction for single applications in homogenous computing environments, these are not applicable to workflows, due to the number and heterogeneity of the involved workflow and system components and their strong interdependencies. In this paper, we investigate workflow performance goals and identify factors that could have a relevant impact. Based on our analysis, we propose a new workflow performance provenance ontology, the Open Provenance Model-based WorkFlow Performance Provenance, or OPM-WFPP, that will enable the empirical study of workflow performance characteristics and variability including complex source attribution.
Enabling structured workflows↗