Search NASA⌕ Search

SEARCH · Search NASA

Results for “communication trace”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

PredCom: A Predictive Approach to Collecting Approximated Communication Traces

Communication traces collected from MPI applications are an important source of information for performance optimization as they can help analysts determine communication patterns and identify inefficiencies. However, their collection, especially at scale, is time consuming, since it usually requires running the complete target application on a large number of nodes. In this work, we present PredCom, a tool-chain to generate a predictive communication proxy based on information gathered from a few small scale runs, which allows us to extract approximate communication traces with an accuracy high enough for most analysis goals. For this, we combine LLVM passes on the original source code (to capture static program structure) with parameter prediction (to capture dynamic and scaling behavior). This approach drastically reduces the time needed for collecting the communication traces, even for traces on large numbers of MPI processes. Here, we demonstrate that PredCom generates communication traces of various applications up to 1612x faster with an accuracy loss of 0.11 on average compared to the original large-scale traces, and we show that the generated traces can be used to optimize process placement.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2010

Scalable Communication Trace Compression

Characterizing the communication behavior of parallel programs through tracing can help understand an application’s characteristics, model its performance, and predict behavior on future systems. However, lossless communication traces can get prohibitively large, causing programmers to resort to variety of other techniques. In this paper, we present a novel approach to lossless communication trace compression. We augment the sequitur compression algorithm to employ it in communication trace compression of parallel programs. We present optimizations to reduce the memory overhead, reduce size of the trace files generated, and enable compression across multiple processes in a parallel program. The evaluation shows improved compression and reduced overhead over other approaches, with up to 3 orders of magnitude improvement for the NAS MG benchmark. We also observe that, unlike existing schemes, the trace files sizes and the memory overhead incurred are less sensitive to, if not independent of, the problem size for the NAS benchmarks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Benchmark Generation and Simulation at Extreme Scale

The path to extreme scale high-performance computing (HPC) poses several challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Investigating the performance of parallel applications at scale on future architectures and the performance impact of different architectural choices is an important component of HPC hardware/software co-design. Simulations using models of future HPC systems and communication traces from applications running on existing HPC systems can offer an insight into the performance of future architectures. This work targets technology developed for scalable application tracing of communication events. It focuses on extreme-scale simulation of HPC applications and their communication behavior via lightweight parallel discrete event simulation for performance estimation and evaluation. Instead of simply replaying a trace within a simulator, this work promotes the generation of a benchmark from traces. This benchmark is subsequently exposed to simulation using models to reflect the performance characteristics of future-generation HPC systems. This technique provides a number of benefits, such as eliminating the data intensive trace replay and enabling simulations at different scales. The presented work features novel software co-design aspects, combining the ScalaTrace tool to generate scalable trace files, the ScalaBenchGen tool to generate the benchmark, and the xSim tool to assess the benchmark characteristics within a simulator.

Lagadapati, Mahesh↗
DOE OSTI2013

Tools for Simulation and Benchmark Generation at Exascale

The path to exascale high-performance computing (HPC) poses several challenges related to power, performance, resilience, productivity, programmability, data movement, and data management. Investigating the performance of parallel applications at scale on future architectures and the performance impact of different architecture choices is an important component of HPC hardware/software co-design. Simulations using models of future HPC systems and communication traces from applications running on existing HPC systems can offer an insight into the performance of future architectures. This work targets technology developed for scalable application tracing of communication events and memory profiles, but can be extended to other areas, such as I/O, control flow, and data flow. It further focuses on extreme-scale simulation of millions of Message Passing Interface (MPI) ranks using a lightweight parallel discrete event simulation (PDES) toolkit for performance evaluation. Instead of simply replaying a trace within a simulation, the approach is to generate a benchmark from it and to run this benchmark within a simulation using models to reflect the performance characteristics of future-generation HPC systems. This provides a number of benefits, such as eliminating the data intensive trace replay and enabling simulations at different scales. The presented work utilizes the ScalaTrace tool to generate scalable trace files, the ScalaBenchGen tool to generate the benchmark, and the xSim tool to run the benchmark within a simulation.

Lagadapati, Mahesh↗
NASA NTRS1991

Synthesizing a protocol converter from executable protocol traces

Communicating finite state machines (CFSM's) with FIFO (first in, first out) queues are used to model a protocol converter. A protocol conversion algorithm is developed and presented for the CSFM model of the protocols A and B. A converter H for protocols A = (A0, A1) and B = (B0, B1) is viewed as a black box such that H is between sender A0 and receiver B1. This gives a resulting protocol X = (A0, H, B1). The conversion algorithm requires a specification of the message relationships between the messages of protocols A and B. It is assumed that protocols A and B have the required progress properties. The algorithm includes a search for related messages from the two protocols in an FIFO from a composite space formed by a Cartesian cross-product of state spaces A1 and B0. The search produces finite-length traces which are combined to form a state machine H, which is examined for freedom from unspecified receptions, deadlocks, and livelocks. A protocol conversion example demonstrates the applicability of the algorithm.

Rajagopal, Murali↗
DOE OSTI2014

The Open Host Network Packet Process Correlator for Windows

The Hone sensors are packet-process correlation engines that log the relationships between applications and the communications they are responsible for. Hone sensors are available for a variety of platforms including Linux, Windows, and MacOSX. Hone sensors are designed to help analysts understand the meaning of communications on a deeper level by associating the origin or destination process to the communication. They do this by tracing communications on a per-packet basis, through the kernel of the operating system to determine their ultimate source/destination on the monitored machine.

Source record↗
DOE OSTI2014

The Open Host Network Packet Process Correlator for Windows

The Hone sensors are packet-process correlation engines that log the relationships between applications and the communications they are responsible for. Hone sensors are available for a variety of platforms including Linux, Windows, and MacOSX. Hone sensors are designed to help analysts understand the meaning of communications on a deeper level by associating the origin or destination process to the communication. They do this by tracing communications on a per-packet basis, through the kernel of the operating system to determine their ultimate source/destination on the monitored machine.

Source record↗
DOE OSTI2015

Static analysis techniques for semiautomatic synthesis of message passing software skeletons

The design of high-performance computing architectures demands performance analysis of large-scale parallel applications to derive various parameters concerning hardware design and software development. The process of performance analysis and benchmarking an application can be done in several ways with varying degrees of fidelity. One of the most cost-effective ways is to do a coarse-grained study of large-scale parallel applications through the use of program skeletons. The concept of a “program skeleton” that we discuss in this article is an abstracted program that is derived from a larger program where source code that is determined to be irrelevant is removed for the purposes of the skeleton. In this work, we develop a semiautomatic approach for extracting program skeletons based on compiler program analysis. Finally, we demonstrate correctness of our skeleton extraction process by comparing details from communication traces, as well as show the performance speedup of using skeletons by running simulations in the SST/macro simulator.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1992

Reducing space overhead for independendent checkpointing

The main disadvantages of independent checkpointing are the possible domino effect and the associated storage space overhead for maintaining multiple checkpoints. In most previous work, it has been assumed that only the checkpoints older than the current global recovery line can be discarded. Here, we generalize a notion of recovery line to potential recovery line. Only the checkpoints belonging to at least one of the potential recovery lines cannot be discarded. By using the model of maximum-sized antichains on a partially ordered set, an efficient algorithm is developed for finding all non-discardable checkpoints, and we show that the number of non-discardable checkpoints cannot exceed N(N+1)/2, where N is the number of processors. Communication trace driven simulation for several hypercube programs is performed to show the benefit of the proposed algorithm for real applications.

Wang, Yi-Min↗
DOE OSTI2011

Probabilistic Communication and I/O Tracing with Deterministic Replay at Scale

With today's petascale supercomputers, applications often exhibit low efficiency, such as poor communication and I/O performance, that can be diagnosed by analysis tools. However, these tools either produce extremely large trace files that complicate performance analysis, or sacrifice accuracy to collect high-level statistical information using crude averaging. This work contributes Scala-H-Trace, which features more aggressive trace compression than any previous approach, particularly for applications that do not show strict regularity in SPMD behavior. Scala-H-Trace uses histograms expressing the probabilistic distribution of arbitrary communication and I/O parameters to capture variations. Yet, where other tools fail to scale, Scala-H-Trace guarantees trace files of near constant size, even for variable communication and I/O patterns, producing trace files orders of magnitudes smaller than using prior approaches. We demonstrate the ability to collect traces of applications running on thousands of processors with the potential to scale well beyond this level. We further present the first approach to deterministically replay such probabilistic traces (a) without deadlocks and (b) in a manner closely resembling the original applications. Our results show either near constant sized traces or only sub-linear increases in trace file sizes irrespective of the number of nodes utilized. Even with the aggressively compressed histogram-based traces, our replay times are within 12% to 15% of the runtime of original codes. Such concise traces resembling the behavior of production-style codes closely and our approach of deterministic replay of probabilistic traces are without precedence.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

Versatile soil gas concentration and isotope monitoring: optimization and integration of novel soil gas probes with online trace gas detection

Abstract. Gas concentrations and isotopic signatures can unveil microbial metabolisms and their responses to environmental changes in soil. Currently, few methods measure in situ soil trace gases such as the products of nitrogen and carbon cycling or volatile organic compounds (VOCs) that constrain microbial biochemical processes like nitrification, methanogenesis, respiration, and microbial communication. Versatile trace gas sampling systems that integrate soil probes with sensitive trace gas analyzers could fill this gap with in situ soil gas measurements that resolve spatial (centimeters) and temporal (minutes) patterns. We developed a system that integrates new porous and hydrophobic sintered polytetrafluoroethylene (sPTFE) diffusive soil gas probes that non-disruptively collect soil gas samples with a transfer system to direct gas from multiple probes to one or more central gas analyzer(s) such as laser and mass spectrometers. Here, we demonstrate the feasibility and versatility of this automated multiprobe system for soil gas measurements of isotopic ratios of nitrous oxide (δ18O, δ15N, and the 15N site preference of N2O), methane, carbon dioxide (δ13C), and VOCs. First, we used an inert silica matrix to challenge probe measurements under controlled gas conditions. By changing and controlling system flow parameters, including the probe flow rate, we optimized recovery of representative soil gas samples while reducing sampling artifacts on subsurface concentrations. Second, we used this system to provide a real-time window into the impact of environmental manipulation of irrigation and soil redox conditions on in situ N2O and VOC concentrations. Moreover, to reveal the dynamics in the stable isotope ratios of N2O (i.e., 14N14N16O, 14N15N16O, 15N14N16O, and 14N14N18O), we developed a new high-precision laser spectrometer with a reduced sample volume demand. Our integrated system – a tunable infrared laser direct absorption spectrometry (TILDAS) in parallel with Vocus proton transfer reaction mass spectrometry (PTR-MS), in line with sPTFE soil gas probes – successfully quantified isotopic signatures for N2O, CO2, and VOCs in real time as responses to changes in the dry–wetting cycle and redox conditions. Broadening the collection of trace gases that can be monitored in the subsurface is critical for monitoring biogeochemical cycles, ecosystem health, and management practices at scales relevant to the soil system.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1992

Lazy checkpoint coordination for bounding rollback propagation

Independent checkpointing allows maximum process autonomy but suffers from potential domino effects. Coordinated checkpointing eliminates the domino effect by sacrificing a certain degree of process autonomy. In this paper, we propose the technique of lazy checkpoint coordination which preserves process autonomy while employing communication-induced checkpoint coordination for bounding rollback propagation. The introduction of the notion of laziness allows a flexible trade-off between the cost for checkpoint coordination and the average rollback distance. Worst-case overhead analysis provides a means for estimating the extra checkpoint overhead. Communication trace-driven simulation for several parallel programs is used to evaluate the benefits of the proposed scheme for real applications.

Wang, Yi-Min↗
NASA NTRS1993

Optimal message log reclamation for independent checkpointing

Independent (uncoordinated) check pointing for parallel and distributed systems allows maximum process autonomy but suffers from possible domino effects and the associated storage space overhead for maintaining multiple checkpoints and message logs. In most research on check pointing and recovery, it was assumed that only the checkpoints and message logs older than the global recovery line can be discarded. It is shown how recovery line transformation and decomposition can be applied to the problem of efficiently identifying all discardable message logs, thereby achieving optimal garbage collection. Communication trace-driven simulation for several parallel programs is used to show the benefits of the proposed algorithm for message log reclamation.

Wang, Yi-Min↗
NASA NTRS1993

Optimal message log reclamation for independent checkpointing

Independent (uncoordinated) check pointing for parallel and distributed systems allows maximum process autonomy but suffers from possible domino effects and the associated storage space overhead for maintaining multiple checkpoints and message logs. In most research on check pointing and recovery, it was assumed that only the checkpoints and message logs older than the global recovery line can be discarded. It is shown how recovery line transformation and decomposition can be applied to the problem of efficiently identifying all discardable message logs, thereby achieving optimal garbage collection. Communication trace-driven simulation for several parallel programs is used to show the benefits of the proposed algorithm for message log reclamation.

Wang, Yi-Min↗
NASA NTRS1992

Scheduling message processing for reducing rollback propagation

Traditional checkpointing and rollback recovery techniques for parallel systems have typically assumed the communication pattern is specified by program behavior. In this paper we exploit the property that the communication pattern can often be changed at run-time without affecting program correctness. A scheduling algorithm for message processing and its implementation for reducing rollback propagation are described. The algorithm incorporates a user-transparent prioritized scheme based upon the run-time communication and checkpointing history. Communication trace-driven simulation for several parallel programs written in the Chare Kernel language demonstrates that the probability of rollback propagation can be reduced at the cost of slight additional performance degradation.

Wang, Yi-Min↗
NASA NTRS1993

Lazy checkpoint coordination for bounding rollback propagation

Independent checkpointing allows maximum process autonomy but suffers from potential domino effects. Coordinated checkpointing eliminates the domino effect by sacrificing a certain degree of process autonomy. In this paper, we propose the technique of lazy checkpoint coordination which preserves process autonomy while employing communication-induced checkpoint coordination for bounding rollback propagation. The introduction of the notion of laziness allows a flexible trade-off between the cost for checkpoint coordination and the average rollback distance. Worst-case overhead analysis provides a means for estimating the extra checkpoint overhead. Communication trace-driven simulation for several parallel programs is used to evaluate the benefits of the proposed scheme for real applications.

Wang, Yi-Min↗
NASA NTRS1994

Optimal message log reclamation for uncoordinated checkpointing

Uncoordinated checkpointing for message-passing systems allows maximum process autonomy and general nondeterministic execution, but suffers from potential domino effect and the large space overhead for maintaining checkpoints and message logs. Traditionally, it has been assumed that only obsolete checkpoints and message logs before the global recovery line can be garbage-collected. Recently, an approach to identifying all garbage checkpoints based on recovery line transformation and decomposition has been developed. We show in this paper that the same approach can be applied to the problem of identifying all garbage message logs for systems requiring message logging to record in-transit messages. Communication trace-driven simulation for several parallel programs is used to evaluate the proposed algorithm.

Wang, Yi-Min↗