Search NASA⌕ Search

SEARCH · Search NASA

Results for “debugging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Production Operations Management System (POMS) for Fermilab Experiments

The Production Operations Management System (POMS) software allows production teams and analysis groups across multiple Fermilab experiments to launch, modify and monitor large scale campaigns of related Monte Carlo or data processing jobs, and currently manages the majority of production computing of experiments at Fermilab.POMS provides a web service interface that enables automated job submission on distributed resources according to customers’ requests, as well as subsequent monitoring of those submissions as well as recovery of failed submissions, debugging and record keeping.POMS interfaces with existing HEP data access, processing, movement and monitoring tools at Fermilab, including Jobsub (for job submission), SAM (for data management), and Landscape (for monitoring), and in combination with them handles the creation of intermediate datasets of files for multistage campaigns.POMS visualizes these campaigns with an interactive GUI interface. An important feature of POMS is a one-to-one connection between the GUI campaign visualizer and a text based representation of the complete campaign configuration. An extensible library of template campaign configurations is available. The templates are user modifiable and map cleanly to and from the GUI description of the campaign.This paper will discuss the current usage, history, and some technical details of POMS, as well as planned future extensions, particularly interfacing with Rucio as well as SAM for file movement.

Mengel, Marc↗
DOE OSTI2024

Multithreading ATLAS offline software: A retrospective

For Run 3, ATLAS redesigned its offline software, Athena, so that the main workflows run completely multithreaded. The resulting substantial reduction in the overall memory requirements allows for better use of machines with many cores. This note will discuss the performance achieved by the multithreaded reconstruction, the process of migrating the large ATLAS code base, and tools and techniques that were useful in debugging threading-related problems.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2014

Study of beam transverse properties of a thermionic electron gun for application to a compact THz free electron laser

A novel thermionic electron gun adopted for use in a high power THz free electron laser (FEL) is proposed in this paper. By optimization of the structural and radiofrequency (RF) parameters, the physical design of the gun is performed using dynamic calculations. Velocity bunching is used to minimize the bunch's energy spread, and the dynamic calculation results indicate that high quality beams can be provided. The transverse properties of the beams generated by the gun are also analyzed. The novel RF focusing effects of the resonance cavity are investigated precisely and are used to establish emittance compensation, which enables the injector length to be reduced. In addition, the causes of the extrema of the beam radius and the normalized transverse emittance are analyzed and interpreted, respectively, and slice simulations are performed to illustrate how the RF focusing varies along the bunch length and to determine the effects of that variation on the emittance compensation. Finally, by observation of the variations of the beam properties in the drift tube behind the electron gun, prospective assembly scenarios for the complete THz-FEL injector are discussed, and a joint-debugging process for the injector is implemented.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

A field programmable gate array-based timing and control system for the dynamic compression sector

A field programmable gate array (FPGA)-based timing and trigger control system has been developed for the Dynamic Compression Sector (DCS) user facility located at the Advanced Photon Source (APS) at Argonne National Laboratory. The DCS is a first-of-its-kind capability dedicated to dynamic compression science. All components of the DCS laser shock station—x-ray choppers, single-shot shutter, internal laser triggers, and shot diagnostics—must be synchronized with respect to the arrival of x rays in the hutch. An FPGA synchronized to the APS storage ring radio frequency clock (352 MHz) generates trigger signals for each stage of the laser and x-ray shutter system with low jitter. The developed FPGA-based control system was the first system used to control the laser and the shutter system since its commissioning, and it has been developing since then to improve the timing jitter. The system is composed of a Zynq FPGA, a debug card, line drivers, and a power supply. The delay and offsets of the trigger signals can be adjusted by using a user-friendly graphical user interface with high precision. Here, the details of the system architecture, timing requirements, firmware, and software implementation along with the performance evaluation are presented in this paper. The system offers low timing jitter (15.5 ps rms) with respect to the APS 352 MHz clock, suitable for the 100 ps (FWHM) x-ray bunch duration at the APS.

47 OTHER INSTRUMENTATION↗
DOE OSTI2018

Methods for analysing the quality of the element base of quantum information technologies

Methods for analysing the quality of the element base of quantum information technologies are considered. Methods for simulating quantum operations with allowance for quantum noise, based on the formalism of quantum operations (operator sum method and Choi – Jamiolkowski isomorphism), are described. Results of simulating some one- and two-qubit quantum transformations with allowance for the amplitude and phase relaxation and the presence of depolarising noise are reported. The quantum state tomography, based on the root approach and maximum likelihood method, is considered. The method of guaranteed estimation of quantum tomography accuracy, which implies quantitative estimation of measurement results, is described. Tomography of a single-qubit transformation on a superconducting quantum processor (IBM) is performed. The practical purpose of the study is to provide high quality and efficiency of quantum information technologies. The results obtained can be used to improve the quality of logic gates of quantum computers and debug the procedures of quantum state control in quantum cryptography problems. The bibliography includes 28 references. (quantum technology)

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2017

Efficient monitoring of CRAB jobs at CMS

CRAB is a tool used for distributed analysis of CMS data. Users can submit sets of jobs with similar requirements (tasks) with a single request. CRAB uses a client-server architecture, where a lightweight client, a server, and ancillary services work together and are maintained by CMS operators at CERN. As with most complex software, good monitoring tools are crucial for efficient use and longterm maintainability. This work gives an overview of the monitoring tools developed to ensure the CRAB server and infrastructure are functional, help operators debug user problems, and minimize overhead and operating cost. This work also illustrates the design choices and gives a report on our experience with the tools we developed and the external ones we used.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2017

Effective HTCondor-based monitoring system for CMS

The CMS experiment at the LHC relies on HTCondor and glideinWMS as its primary batch and pilot-based Grid provisioning systems, respectively. Given the scale of the global queue in CMS, the operators found it increasingly difficult to monitor the pool to find problems and fix them. The operators had to rely on several different web pages, with several different levels of information, and sift tirelessly through log files in order to monitor the pool completely. Therefore, coming up with a suitable monitoring system was one of the crucial items before the beginning of the LHC Run 2 in order to ensure early detection of issues and to give a good overview of the whole pool. Our new monitoring page utilizes the HTCondor ClassAd information to provide a complete picture of the whole submission infrastructure in CMS. The monitoring page includes useful information from HTCondor schedulers, central managers, the glideinWMS frontend, and factories. It also incorporates information about users and tasks making it easy for operators to provide support and debug issues.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Benchmarking characterization methods for noisy quantum circuits

Effective methods for characterizing the noise in quantum computing devices are essential for programming and debugging circuit performance. Existing approaches vary in the information obtained as well as the amount of quantum and classical resources required, with more information generally requiring more resources. Here we benchmark the characterization methods of gate set tomography, Pauli channel noise reconstruction, and empirical direct characterization for developing models that describe noisy quantum circuit performance on a 27-qubit superconducting transmon device. We evaluate these models by comparing the accuracy of noisy circuit simulations with the corresponding experimental observations. Here, we find that the agreement of noise model to experiment does not correlate with the information gained by characterization and that the underlying circuit strongly influences the best choice of characterization approach. Empirical direct characterization scales best of the methods we tested and produced the most accurate characterizations across our benchmarks.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Distributed Order Recording Techniques for Efficient Record-and-Replay of Multi-threaded Programs

After all these years and all these other shared memory programming frameworks, OpenMP is still the most popular one. However, its greater levels of non-deterministic execution makes debugging and testing more challenging. The ability to record and deterministically replay the program execution is key to address this challenge. However, scalably replaying OpenMP programs is still an unresolved problem. In this paper, we propose two novel techniques that use Distributed Clock (DC) and Distributed Epoch (DE) recording schemes to eliminate excessive thread synchronization for OpenMP record and replay. Our evaluation on representative HPC applications with ReOMP, which we used to realize DC and DE recording, shows that our approach is 2-5x more efficient than traditional approaches that synchronize on every shared-memory access. Furthermore, we demonstrate that our approach can be easily combined with MPI-level replay tools to replay non-trivial MPI+OpenMP applications. We achieve this by integrating ReOMP into ReMPI, an existing scalable MPI record-and-replay tool, with only a small MPI-scale-independent runtime overhead.

Fu, Xiang↗
DOE OSTI2021

Invited: Bambu: an Open-Source Research Framework for the High-Level Synthesis of Complex Applications

This paper presents the open-source High-Level Synthesis research framework Bambu. The framework provides an open-source starting point to experiment with new ideas across High-Level Synthesis, high-level verification and debugging, FPGA/ASIC design, design flow space exploration, and parallel hardware accelerator design. The tool accepts as input standard C/C++ specifications and compiler intermediate representations (IRs) coming from the well-known Clang/LLVM and GCC com- pilers. The broad spectrum and flexibility of input formats allow the electronic design automation (EDA) research community to explore and integrate new transformations and optimizations. The easily extendable modular framework already includes many op- timizations and HLS benchmarks. The integration with synthesis and verification backends (commercial and open-source) allows researchers to quickly test any new finding and easily obtain performance and resource usage metrics for a given application. Different FPGA devices are supported from several different vendors: AMD/XILINX, Intel/Altera, Lattice Semiconductor, and NanoXplore. Finally, integration with the OpenRoad open-source end-to-end silicon compiler perfectly fits with the recent push towards open-source EDA.

Ferrandi, Fabrizio↗
DOE OSTI2021

A Remote Development Process and Platform for Power Electronic Systems

Like many areas in research and development, power electronic and distributed energy resource research and development slowed during the COVID pandemic. While the presence of the virus has reduced and workers are returning to shared offices, uncertainty of potential future challenges are still present. This work proposes a development and testing network that provides researchers the ability to operate remotely and coordinate control and debug activities for power electronic systems. This development and testing network has been demonstrated in both controller hardware-in-the-loop (C-HIL) and hardware testing.

Starke, Michael↗
DOE OSTI2019

Scalable Performance Awareness for In Situ Scientific Applications

Part of the promise of exascale computing and the next generation of scientific simulation codes is the ability to bring together time and spatial scales that have traditionally been treated separately. This enables creating complex coupled simulations and in situ analysis pipelines, encompassing such things as "whole device" fusion models or the simulation of cities from sewers to rooftops. Unfortunately, the HPC analysis tools that have been built up over the preceding decades are ill suited to the debugging and performance analysis of such computational ensembles. In this paper, we present a new vision for performance measurement and understanding of HPC codes, MonitoringAnalytics (MONA). MONA is designed to be a flexible, high performance monitoring infrastructure that can perform monitoring analysis in place or in transit by embedding analytics and characterization directly into the data stream, without relying upon delivering all monitoring information to a central database for post-processing. It addresses the trade-offs between the prohibitively expensive capture of all performance characteristics and not capturing enough to detect the features of interest. We demonstrate several uses of MONA; capturing and indexing multi-executable performance profiles to enable later processing, extraction of performance primitives to enable the generation of customizable benchmarks and performance skeletons, and extracting communication and application behaviors to enable better control and placement for the current and future runs of the science ensemble. Relevant performance information based on a system for MONA built from ADIOS and SOSflow technologies is provided for DOE science applications and leadership machines.

Wolf, Matthew D.↗
DOE OSTI2014

Online Monitoring System for Performance Fault Detection

To achieve the exaFLOPS performance within a contain power budget, next supercomputers will feature hundreds of millions of components operating at low- and near-threshold voltage. As the probability that at least one of these components fails during the execution of an application approaches certainty, it seems unrealistic to expect that any run of a scientific application will not experience some performance faults. We believe that there is need of a new generation of light-weight performance and debugging tools that can be used online even during production runs of parallel applications and that can identify performance anomalies during the application execution. In this work we propose the design and implementation of a monitoring system that continuously inspects the evolution of run

HPC↗
DOE OSTI2016

Asynchronous Runtimes in Action: An Introspective Framework for a Next Gen Runtime

One of the most critical challenges that new high performance systems face is the lack of system software support for these large scale systems. Investment on system stack components is essential in the development, debugging and optimization of the new emerging programming models. These emerging models have the promise to better utilize the vast hardware resources available in current and future systems. To aid in the development of applications and new system stacks, runtimes, as instances of their respective execution models, need to produce facilities to introspect their inner workings and allow an indepth attribution of performance bottlenecks and computational patterns. In other words, the runtime systems need to reduce their opacity to observers so that users of a novel program execution model can adapt their designs to fit the intended model usage, regardless of the layer that they are working on. This design/development loop (akin to co-design) enables synergistic opportunities across the entire computational stack. This paper presents the design and implementation of a simple “gray” box performance attribution harness running inside a fine grain runtime system: the Open Community Runtime (OCR). We showcase what such a framework can indicate regarding the runtime behavior while running at scale. To this end, we have designed a set of synthetic scenarios aimed to test the runtime at their best and worst cases. We present an analysis of the most important runtime features, properties and idiosyncrasies that will affect the development of new runtime features, algorithmic selection, and application development.

high performance distributed runtimes↗
DOE OSTI2018

The Ongoing Evolution of OpenMP

Here, this paper presents an overview of the past, present and future of the OpenMP application programming interface (API). While the API originally specified a small set of directives that guided shared memory fork-join parallelization of loops and program sections, OpenMP now provides a richer set of directives that capture a wide range of parallelization strategies that are not strictly limited to shared memory. As we look toward the future of OpenMP, we immediately see further evolution of the support for that range of parallelization strategies and the addition of direct support for debugging and performance analysis tools. Looking beyond the next major release of the specification of the OpenMP API, we expect the specification eventually to include support for more parallelization strategies and to embrace closer integration into its Fortran, C and, in particular, C++ base languages, which will likely require the API to adopt additional programming abstractions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Upgrade of the Minos+ Experiment Data Acquisition for the High Energy NuMI Beam Run

The Minos+ experiment is an extension of the Minos experiment at a higher energy and more intense neutrino beam, with the data collection having begun in the fall of 2013. The neutrino beam is provided by the Neutrinos from the Main Injector (NuMI) beam-line at Fermi National Accelerator Laboratory (Fermilab). The detector apparatus consists of two main detectors, one underground at Fermilab and the other in Soudan, Minnesota with the purpose of studying neutrino oscillations at a base line of 735 km. The original data acquisition system has been running for several years collecting data from NuMI, but with the extended run from 2013, parts of the system needed to be replaced due to obsolescence, reliability problems, and data throughput limitations. Specifically, we have replaced the front-end readout controllers, event builder, and data acquisition computing and trigger processing farms with modern, modular and reliable devices with few single points of failure. The new system is based on gigabit Ethernet TCP/IP communication to implement the event building and concatenation of data from many front-end VME readout crates. The simplicity and partitionability of the new system greatly eases the debugging and diagnosing process. As a result, the new system improves throughput by about a factor of three compared to the old system, up to 800 megabits per second, and has proven robust and reliable in the current run.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2016

PATHA: Performance Analysis Tool for HPC Applications

Large science projects rely on complex workflows to analyze terabytes or petabytes of data. These jobs are often running over thousands of CPU cores and simultaneously performing data accesses, data movements, and computation. It is difficult to identify bottlenecks or to debug the performance issues in these large workflows. In order to address these challenges, we have developed Performance Analysis Tool for HPC Applications (PATHA) using the state-of-art open source big data processing tools. Our framework can ingest system logs to extract key performance measures, and apply the most sophisticated statistical tools and data mining methods on the performance data. Furthermore, it utilizes an efficient data processing engine to allow users to interactively analyze a large amount of different types of logs and measurements. To illustrate the functionality of PATHA, we conduct a case study on the workflows from an astronomy project known as the Palomar Transient Factory (PTF). This study processed 1.6 TB of system logs collected on the NERSC supercomputer Edison. Using PATHA, we were able to identify performance bottlenecks, which reside in three tasks of PTF workflow with the dependency on the density of celestial objects.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2024

Impacts of floating-point non-associativity on reproducibility for HPC and deep learning applications

Run to run variability in parallel programs caused by floating-point non-associativity has been known to significantly affect reproducibility in iterative algorithms, due to accumulating errors. Non-reproducibility can critically affect the efficiency and effectiveness of correctness testing for stochastic programs. Recently, the sensitivity of deep learning training and inference pipelines to floating-point non-associativity has been found to sometimes be extreme. It can prevent certification for commercial applications, accurate assessment of robustness and sensitivity, and bug detection. New approaches in scientific computing applications have coupled deep learning models with high-performance computing, leading to an aggravation of debugging and testing challenges. Here we perform an investigation of the statistical properties of floating-point non-associativity within modern parallel programming models, and analyze performance and productivity impacts of replacing atomic operations with deterministic alternatives on GPUs. We examine the recently-added deterministic options in PyTorch within the context of GPU deployment for deep learning, uncovering and quantifying the impacts of input parameters triggering run to run variability and reporting on the reliability and completeness of the documentation. Finally, we evaluate the strategy of exploiting automatic determinism that could be provided by deterministic hardware, using the Groq LPUTM accelerator for inference portions of the deep learning pipeline. We demonstrate the benefits that a hardware-based strategy can provide within reproducibility and correctness efforts.

Shanmugavelu, Sanjif↗