Particle-in-cell (hPIC) data
Data generated from the hPIC particle-in-cell code to construct a surrogate ion energy-angle distribution (IEAD) model.
Engineering topics
Publications and source records attributed to Bernholdt, David E.
Data generated from the hPIC particle-in-cell code to construct a surrogate ion energy-angle distribution (IEAD) model.
In the world of high-performance computing, fault tolerance and application resilience are becoming some of the primary concerns because of increasing hardware failures and memory corruptions. While the research community has been investigating various options, from system-level solutions to application-level solutions, standards such as the Message Passing Interface (MPI) are also starting to include such capabilities. The current proposal for MPI fault tolerant is centered around the User-Level Failure Mitigation (ULFM) concept, which provides means for fault detection and recovery of the MPI layer. This approach does not address application-level recovery, which is currently left to application developers. In this work, we present a mod- ification of some of the benchmarks of the NAS parallel benchmark (NPB) to include support of the ULFM capabilities as well as application-level strategies and mechanisms for application-level failure recovery. As such, we present: (i) an application-level library to checkpoint and restore data, (ii) extensions of NPB benchmarks for fault tolerance based on different strategies, (iii) a fault injection tool, and (iv) some preliminary results that show the impact of such fault tolerant strategies on the application execution.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
The multicore generation of scientific high performance computing has provided a platform for the realization of Exascale computing, and has also underscored the need for new paradigms in coding parallel applications. The current standard for writing parallel applications requires programmers to use languages designed for sequential execution. These languages have abstractions that only allow programmers to operate on the process centric local view of data. To provide suitable languages for parallel execution, many research efforts have designed languages based on the Partitioned Global Address Space (PGAS) programming model. Chapel is one of the more recent languages to be developed using this model. Chapel supports multithreaded execution with high-level abstractions for parallelism. With Chapel in mind, we have developed a set of directives that serve as intermediate expressions for transitioning scientific applications from languages designed for sequential execution to PGAS languages like Chapel that are being developed with parallelism in mind.
Abstract Not Provided
Explore the source record for details and available documents.
Programming with communication libraries such as the Message Passing Interface (MPI) obscures the high-level intent of the communication in an application and makes static communication analysis difficult to do. Compilers are unaware of communication libraries specifics, leading to the exclusion of communication patterns from any automated analysis and optimizations. To overcome this, communication patterns can be expressed at higher-levels of abstraction and incrementally added to existing MPI applications. In this paper, we propose the use of directives to clearly express the communication intent of an application in a way that is not specific to a given communication library. Our communication directives allow programmers to express communication among processes in a portable way, giving hints to the compiler on regions of computations that can be overlapped with communication and relaxing communication constraints on the ordering, completion and synchronization of the communication imposed by specific libraries such as MPI. The directives can then be translated by the compiler into message passing calls that efficiently implement the intended pattern and be targeted to multiple communication libraries. Thus far, we have used the directives to express point-to-point communication patterns in C, C++ and Fortran applications, and have translated them to MPI and SHMEM.
The increasing availability of large scale computing capabilities has accelerated the development of high-fidelity coupled simulations. Such simulations typically involve the integration of models that implement various aspects of the complex phenomena under investigation. Coupled simulations are playing an integral role in fields such as climate modeling, earth systems modeling, rocket simulations, computational chemistry, fusion research, and many other computational fields. Model coupling provides scientists with systematic ways to virtually explore the physical, mathematical, and computational aspects of the problem. Such exploration is rarely done using a single execution of a simulation, but rather by aggregating the results from many simulation runs that, together, serve to bring to light novel knowledge about the system under investigation. Furthermore, it is often the case (particularly in engineering disciplines) that the study of the underlying system takes the form of an optimization regime, where the control parameter space is explored to optimize an objective functions that captures system realizability, cost, performance, or a combination thereof. Novel and flexible frameworks that facilitate the integration of the disparate models into a holistic simulation are used to perform this research, while making efficient use of the available computational resources. In this paper, we describe the integration of the DAKOTA optimization and parameter sweep toolkit with the Integrated Plasma Simulator (IPS), a component-based framework for loosely coupled simulations. The integration allows DAKOTA to exploit the internal task and resource management of the IPS to dynamically instantiate simulation instances within a single IPS instance, allowing for greater control over the trade-off between efficiency of resource utilization and time to completion. We present a case study showing the use of the combined DAKOTA-IPS system to aid in the design of a lithium ion battery (LIB) cell, by studying a coupled system involving the electrochemistry and ion transport at the lower length scales and thermal energy transport at the device scales. The DAKOTA-IPS system provides a flexible tool for use in optimization and parameter sweep studies involving loosely coupled simulations that is suitable for use in situations where changes to the constituent components in the coupled simulation are impractical due to intellectual property or code heritage issues.
The vision of the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program is to bring truly predictive modeling and simulation (M&S) capabilities to the nuclear engineering community in order to enable a new approach to the design and analysis of nuclear energy systems. From its inception, the NEAMS program has always envisioned a broad user base for its software and scientific products, including researchers within the DOE complex, nuclear industry technology developers and vendors, and operators. However activities to date have focused almost exclusively on interactions with NEAMS sponsors, who are also near-term users of NEAMS technologies. The task of the NEAMS Capability Transfer (CT) program element for FY2011 is to develop a comprehensive plan to support the program's needs for user outreach and technology transfer. In order to obtain community input to this plan, a 'NEAMS Capability Transfer Roadmapping Workshop' was held 4-5 April 2011 in Chattanooga, TN, and is summarized in this report. The 30 workshop participants represented the NEAMS program, the DOE and industrial user communities, and several outside programs. The workshop included a series of presentations providing an overview of the NEAMS program and presentations on the user outreach and technology transfer experiences of (1) The Advanced Simulation and Computing (ASC) program, (2) The Standardized Computer Analysis for Licensing Evaluation (SCALE) project, and (3) The Consortium for Advanced Simulation of Light Water Reactors (CASL), followed by discussion sessions. Based on the workshop and other discussions throughout the year, we make a number of recommendations of key areas for the NEAMS program to develop the user outreach and technology transfer activities: (1) Engage not only DOE, but also industrial users sooner and more often; (2) Engage with the Nuclear Regulatory Commission to facilitate their understanding and acceptance of NEAMS approach to predictive M&S; (3) Place requirements gathering from prospective users on a more formal footing, updating requirements on a regular basis and incorporate them into planning and execution of the project in a traceable fashion; (4) Seek out the best available data for validation purposes, and work with experimental programs to design and carry out new experiments that satisfy the need for data suitable for validation of high-fidelity M&S codes; (5) Develop and implement program-wide plans and policies for export control, licensing, and distribution of NEAMS software products; (6) Establish a program of sponsored alpha testing by experienced users in order to obtain feedback on NEAMS codes; (7) Provide technical support for NEAMS software products; (8) Develop and deliver documentation, tutorial materials, and live training classes; and (9) Be prepared to support outside users who wish to contribute to the codes.
This paper discusses on-going work with the Integrated Plasma Simulator (IPS), a framework for coupled multiphysics simulations of plasmas, to allow simulations to run through the loss of nodes on which the simulation is executing. While many different techniques are available to improve the fault tolerance of computational science applications on high-performance computer systems, checkpoint/restart (C/R) remains virtually the only one that see widespread use in practice. Our focus here is to augment the traditional C/R approach with additional techniques that can provide a more localized and tailored response to faults based on the ability to restart failed tasks on an individual basis, and the use of information external to the application itself in order to guide decision-making, in many cases avoiding the need to stop and restart the entire simulation. This capability involves several features within the IPS framework, and leverages the Fault Tolerance Backplane, a publish/subscribe event service to disseminate fault-related information throughout HPC systems, to obtain information from the Reliability, Availability and Serviceability (RAS) subsystem of the HPC system. This work is described in the context of Cray XT-series computer systems for concreteness, but is applicable to other environments as well. As part of the analysis of this work, we discuss the requirements to generalize this approach to other complex simulation applications beyond the Integrated Plasma Simulator.
Parareal is a novel algorithm that allows the solution of time-dependent systems of differential or partial differential equations (PDE) to be parallelized in the temporal domain. Parareal-based implementations of PDE problems can take advantage of this parallelism to significantly reduce the time to solution for a simulation (though at an increased total cost) while making effective use of the much larger processor counts available on current high-end systems. In this paper, we present a dynamic, dependency-driven version of the parareal algorithm which breaks the final sequential bottleneck remaining in the original formulation, making it amenable to a "many-task" treatment. We further improve the cost and execution time of the algorithm by introducing a moving window for time slices, which avoids the execution of tasks which contribute little to the final global solution. We describe how this approach has been realized in the Integrated Plasma Simulator (IPS), a framework for coupled multiphysics simulations, and examine the trade-offs among time-to-solution, total cost, and resource utilization efficiency as a function of the compute resources applied to the problem.
No abstract prepared.
Many modern scientific applications, which are designed to utilize high performance parallel com- puters, occupy hundreds of thousands of computational cores running for days or even weeks. Since many scien- tists compete for resources, most supercomputing centers practice strict scheduling policies and perform meticulous accounting on their usage. Thus computing resources and time assigned to a user is considered invaluable. However, most applications are not well prepared for un- foreseeable faults, still relying on primitive fault tolerance techniques. Considering that ever-plunging mean time to interrupt (MTTI) is making scientific applications more vulnerable to faults, it is increasingly important to provide users not only an improved fault tolerant environment, but also a framework to support their own fault tolerance policies so that their allocation times can be best utilized. This paper addresses a user level fault tolerance policy management based on a holistic approach to digest and correlate fault related information. It introduces simple semantics with which users express their policies on faults, and illustrates how event correlation techniques can be applied to manage and determine the most preferable user policies. The paper also discusses an implementation of the framework using open source software, and demonstrates, as an example, how a molecular dynamics simulation application running on the institutional cluster at Oak Ridge National Laboratory benefits from it.
This paper discusses the Integrated Plasma Simulator (IPS), a framework for coupled multiphysics simulation of fusion plasmas, in the context of many-task computing. The IPS supports multiple levels of parallelism: individual computational tasks can be parallel, components can launch multiple tasks concurrently, tasks from multiple components can be executed concurrently within a simulation, and multiple simulations can be run simultaneously. Each level of parallelism is constructed on top of the many-task computing capabilities implemented in the IPS, the foundation for the parallelism presented at the multiple simulation level. We show that a modest number of simultaneous simulations, with appropriately sized resource allocations, can provide a better trade-off between resource utilization and overall execution time than if they are run as separate jobs. This approach is highly beneficial for situations in which individual simulation tasks may differ significantly in parallel scalability, as is the case in many scientific communities where coupled simulations rely substantially on legacy code.
As computing capabilities have increased, the coupling of computational models has become an increasingly viable and therefore important way of improving the physical fidelity of simulations. Applications currently using some form of multicode or multi-component coupling include climate modeling, rocket simulations, and chemistry. In recent years, the plasma physics community has also begun to pursue integrated multiphysics simulations for space weather and fusion energy applications. Such model coupling generally exposes new issues in the physical, mathematical, and computational aspects of the problem. This paper focuses on the computational aspects of one such effort, detailing the design, and implementation of the Integrated Plasma Simulator (IPS) for the Center for Simulation of Wave Interactions with Magnetohydrodynamics (SWIM). The IPS framework focuses on maximizing flexibility for the creators of loosely-coupled component-based simulations, and provides services for execution coordination, resource management, data management, and inter-component communication. It also serves as a proving ground for a concurrent 'multi-tasking' execution model to improve resource utilization, and application-level fault tolerance. We also briefly describe how the IPS has been applied to several problems of interest to the fusion community.
Complex tensor contraction expressions arise in accurate electronic structure models in quantum chemistry, such as the coupled cluster method. This paper addresses two complementary aspects of performance optimization of such tensor contraction expressions. Transformations using algebraic properties of commutativity and associativity can be used to significantly decrease the number of arithmetic operations required for evaluation of these expressions. The identification of common subexpressions among a set of tensor contraction expressions can result in a reduction of the total number of operations required to evaluate the tensor contractions. The first part of the paper describes an effective algorithm for operation minimization with common subexpression identification and demonstrates its effectiveness on tensor contraction expressions for coupled cluster equations. The second part of the paper highlights the importance of data layout transformation in the optimization of tensor contraction computations on modern processors. A number of considerations such as minimization of cache misses and utilization of multimedia vector instructions are discussed. A library for efficient index permutation of multi-dimensional tensors is described and experimental performance data is provided that demonstrates its effectiveness.