Search NASASearch

SEARCH · Search NASA

Results for “Debugging”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

36 records · Page 2

Using containers to speed up development, to run integration tests and to teach about distributed systems

GlideinWMS is a workload manager provisioning resources for many experiments including CMS and DUNE. The software is distributed both as native packages and specialized production containers. Following an approach used in other communities like web development we built our workspaces, system-like containers to ease development and testing. Developers can change the source tree or check out a different branch and quickly reconfigure the services to see the effect of their changes. In this paper, we’ll talk about what differentiates workspaces from other containers. We’ll describe our base system composed of three containers. A one-node cluster including a compute element and a batch system. A GlideinWMS Factory controlling pilot jobs. And a scheduler and Frontend, to submit jobs and provision resources. Additional containers can be used for optional components. This system can easily run on a laptop and we’ll share our evaluation of different container runtimes, with an eye for ease of use and performance. Finally, we’ll talk about our experience as developers and with students. The GlideinWMS workspaces are easily integrated with IDEs like VS Code, simplifying debugging and allowing development and testing of the system also when offline. They simplified the training and onboarding of new team members and Summer interns. And they were useful in workshops where students could have first-hand experience with the mechanisms and components that, in production, run millions of jobs.

Mambelli, Marco

Zero Emission Cargo Transport (ZECT) II Demonstration: South Coast Air Quality Management District (Final Report)

The South Coast Air Quality Management District (South Coast AQMD), California Air Resources Board (CARB) and Southern California Association of Governments (SCAG) — the agencies responsible for preparing the State Implementation Plan required under the federal Clean Air Act — have agreed that attainment of federal air quality standards for the region will require a transition to the broad use of zero and near-zero emission energy sources in cars, trucks and other equipment. Accordingly, the 2012 South Coast AQMD Air Quality Management Plan, the SCAG 2012 Regional Transportation Plan, and the “Vision for Clean Air: A Framework for Air Quality and Climate Control Planning” all identify the need to immediately enact a phasing in of zero and near-zero emission technologies to meet air quality goals. In 2014, South Coast AQMD was awarded grant funding under the US Department of Energy Zero Emission Cargo Transport (ZECT) II Demonstration program to develop and demonstrate zero-emission drayage trucks for goods movement operations between the Port of Los Angeles (POLA) and Port of Long Beach (POLB) near dock rail yards and warehouses: 1) development and demonstration of zero-emission fuel cell range extended electric drayage trucks and 2) development and demonstration of hybrid electric drayage trucks. The purpose of this project was to accelerate deployment of zero emission cargo transport technologies to reduce harmful diesel emissions, petroleum consumption and greenhouse gases in the surrounding communities along the goods movement corridors that are impacted by heavy diesel traffic and the associated air pollution. Between 2014 – 2024, six ZECT II zero-emission fuel cell drayage truck platforms, including fuel cell range extended and CNG hybrid trucks, were successfully designed, developed, integrated, built, tested, and demonstrated with drayage fleet operators in transportation corridors within areas of the South Coast AQMD jurisdiction in Southern California such as in and around POLA and POLB. Portable hydrogen refueling was deployed to support the fuel cell vehicles. The project had real-time improvement with on-going debugging and optimizations while the vehicles were under demonstration. All platforms demonstrated sufficient or excess power, torque, and energy to support 82,000lbs Gross Vehicle Weight Rating and gradeability to perform their daily duty cycles. Collectively, the trucks drove over 23,000 miles during their respective demonstration phases. The ZECT II project was the first of its kind to demonstrate the commercial viability that supported the additional technology breakthroughs for Class 8 zero emission trucks and validations as well as the regulatory basis for all the zero-emission regulation that we know today, such as the Innovative Clean Transit regulation, Advanced Clean Trucks and Clean Fleet regulations.

08 HYDROGEN

Code Coverage Status of the ARC Code DIF3D

The Argonne Reactor Code (ARC) software system supports users in their fast reactor design goals by providing neutronic, thermal-hydraulic, and structural analysis capabilities. DIF3D plays a pivotal role in the ARC system as the primary homogenized assembly neutronic calculation methodology for fast reactor problems. Over its 40 years history, ARC software usage with DIF3D has been applied to numerous fast and thermal spectrum reactor analysis projects with good to excellent comparison against experiments. With continued improvement of computation resources, many of the geometry modeling capabilities in DIF3D that were primarily used in low order schemes are not really needed anymore. Today, the diffusion and transport capabilities of DIF3D-VARIANT are primarily used in the reactor design process with some scattered usage of DIF3D-FD and DIF3D-Nodal. In recent work, the DIF3D software verification was completed for DIF3D-FD and DIF3D-VARIANT on the geometry options used in the Versatile Test Reactor project. While we can be confident that these capabilities of DIF3D are well used and thus trusted, it does not demonstrate that all possible input options of DIF3D are actually working, but just those that were tested as part of VTR are and that they are correct. Thus, the purpose of the present work is to identify a set of test problems for DIF3D and assess the code coverage of DIF3D for those test problems. The goal is to document what parts of the existing DIF3D code are touched by the set of test problems and which are not. Because the verification work done on DIF3D-VARIANT and DIF3D-FD was focused on the most common uses of DIF3D for fast reactor analysis, the code coverage assessment of those capabilities is the highest priority. This will ensure that nothing is being missed by the existing verification test problems that DIF3D relies upon. The DIF3D-Nodal capability will also be inspected for code coverage as part of this work to further ensure that regular regression testing of DIF3D will trap any likely errors the end user might experience with the DIF3D software. The code coverage analysis of DIF3D was performed with the Code Coverage Tool of the Intel Fortran compiler which requires modifications to the compilation of DIF3D. The detailed coverage tables are given for each submodule of DIF3D separately, and for the submodules which are primarily developed for DIF3D, most of the source files could be at least partially touched. Most of the uncovered parts/files could be easily ignored, because they are either for error message and debugging output or obviously not needed by DIF3D. Out of the entire source codes of DIF3D, only a few uncovered modules deserve further investigation.

22 GENERAL STUDIES OF NUCLEAR REACTORS

Code Coverage Status of the ARC Code PERSENT

The Argonne Reactor Code (ARC) software system supports users in their fast reactor design goals by providing neutronic, thermal-hydraulic, and structural analysis capabilities. PERSENT fulfills the role of generating reactivity coefficients for a given time point of a REBUS calculation usable in a point kinetics based safety analysis capability. PERSENT also provides a sensitivity coefficient capability on eigenvalue, reactivity worth, and several other key coefficients that are used in the follow-on safety analysis. Given a co-variance matrix, PERSENT can carry out the uncertainty quantification to indicate the amount of error in the reactivity coefficients derived from the errors in the cross section measurements. With continued improvement of computational resources, many of the geometry modeling capabilities in DIF3D that were primarily used in low order schemes are not really needed anymore. Today, the diffusion and transport capabilities of DIF3D-VARIANT are primarily used in the reactor design process with some scattered usage of DIF3D-FD and DIF3D-Nodal. PERSENT is part of the ARC code system and is built around DIF3D-VARIANT and the flux solution it provides. The purpose of the present work is to identify a set of test problems for PERSENT and assess the code coverage of PERSENT for those test problems. PERSENT treats the DIF3D executable as an external executable and thus the code coverage considerations only need to focus on the PERSENT source code and only a fraction of the connected modules in the existing ARC software library. The goal is to document what parts of the existing PERSENT code are touched by the set of test problems and which are not. Because the verification work done on PERSENT was focused on the most common uses of PERSENT for fast reactor analysis, the code coverage assessment of those capabilities is the highest priority. This will ensure that nothing is being missed by the existing verification test problems that users of PERSENT rely upon. The code coverage analysis of PERSENT was performed with the Code Coverage Tool of the Intel Fortran compiler which requires modifications to the compilation of PERSENT. The detailed coverage tables are given for each submodule of PERSENT. Most of the uncovered parts/files could be easily ignored because they are either for error message and debugging output or not needed by PERSENT today. Only a few uncovered parts of PERSENT deserve extending the verification test suite.

22 GENERAL STUDIES OF NUCLEAR REACTORS

Code Coverage Status of the ARC code GAMSRC

The Argonne Reactor Code (ARC) software system supports users in their fast reactor design goals by providing neutronic, thermal-hydraulic, and structural analysis capabilities. GAMSOR serves as procedure to obtain the neutron and gamma power distribution information in the ARC code system. GAMSOR is a specially modified version of DIF3D (dif3d.x becomes dif3d_gamsor.x). After some work, it was determined that carrying out software on GAMSOR was impractical and would not fit well with commercial grade dedication. GAMSRC was written in the last decade to replace GAMSOR such that the user can rely soley upon the verified DIF3D (dif3d.x) code and GAMSRC to obtain the neutron and gamma power distribution information. GAMSRC is also fully verified and ready for commercial grade dedication. In the coming years, GAMSOR will be deprecated and GAMSRC will fully take over in the ARC code system. This document identifies the set of test problems used to assess the code coverage for GAMSRC. The goal is to document what parts of the existing GAMSRC code are touched by the set of test problems and which are not. The code coverage analysis of GAMSRC was performed with the Code Coverage Tool of the Intel Fortran compiler which requires modifications to the compilation of GAMSRC. The code coverage tables are given for each submodule of GAMSRC. Because GAMSRC links to modules in DIF3D, some details on coverage changes to the DIF3D lined files is provided. As will be seen, most of the uncovered parts/files can be ignored because they are either for error message and debugging output or obviously not needed by GAMSRC today.

22 GENERAL STUDIES OF NUCLEAR REACTORS

Code Coverage Status of the ARC Code DASSH-F

The Argonne Reactor Code (ARC) software system supports users in their fast reactor design goals by providing neutronic, thermal-hydraulic, and structural analysis capabilities. DASSH-F serves as a steady state thermal hydraulic capability within the ARC system and replaces the SE2-ANL software that preceded it. This document identifies the set of test problems used to assess the code coverage for DASSH-F. The goal is to document what parts of the existing DASSH-F code are touched by the set of test problems and which are not. Because the verification work remains to be done on DASSH-F, one can assume that most of these issues will be resolved as part of that work. The code coverage analysis of DASSH-F was performed with the Code Coverage Tool of the Intel Fortran compiler which requires modifications to the compilation of DASSH-F. The code coverage tables are given for each submodule of DASSH-F. Because DASSH-F links to modules in DIF3D, some details on coverage changes to the DIF3D linked files is provided. As will be seen, most of the uncovered parts/files can be ignored because they are either for error message and debugging output or obviously not needed by DASSH-F today. Seven features of the DASSH-F code were identified to not be covered by the existing testing suite and thus additional verification test problems are suggested to fully cover these sections.

22 GENERAL STUDIES OF NUCLEAR REACTORS

RAPIDS2: A SciDAC Institute for Computer Science, Data, and Artificial Intelligence (University of Delaware)

This report summarizes the activities, technical accomplishments, and outcomes of the RAPIDS2 Institute project at the University of Delaware (UD). The RAPIDS2 Institute was a large multi-institution project with the objective of assisting SciDAC and Office of Science application teams in the use of DOE supercomputing resources to achieve scientific breakthroughs. The UD team contributed to this effort through work on formal software verification. This thrust aims to reduce software developer time and effort, especially regarding debugging and testing, and to increase confidence in the correctness of the results computed by the software.

97 MATHEMATICS AND COMPUTING

Advancing $otsdaq$ for Optimized Data Acquisition

High-energy physics (HEP) experiments demand data acquisition (DAQ) systems capable of orchestrating complex detector operations, high data throughput, and responsive, real-time feedback. Traditional systems often have steep learning curves, making onboarding difficult for new users. The Off-The-Shelf Data Acquisition $otsdaq$ framework was developed to address these issues by providing a modular and flexible interface that is easier to operate while remaining customizable enough for experimental setups. As the upcoming Mu2e experiment prepares for deployment, improving stability, usability, and performance has become increasingly critical. Our work enhances $otsdaq$ with features that streamline visualization, correct data metrics, improve debugging workflows, and stabilize the user interface.

Mohammed, Ali (ORCID:0009000860386626)

DUNE Rucio Server Scalabiilty Studies

The DUNE collaboration has an ongoing production effort to simulate the full detectors and to analyze the various prototypes that are currently running. Rucio is used to manage the 40PB of files made to date. When 500 or more jobs were sending output to Rucio simultaneously via Rucio upload, we observed timeouts, unhandled exceptions, and Rucio server restarts due to slow performance. In collaboration with the core Rucio team we did a full review of the Rucio upload code and identified several optimizations that can be made. We also have deployed the Ingress load balancer in front of our Rucio servers and added a database connection pooling utility. These changes led to significant improvement both in reliability and scalability, yet we anticipate even better performance will eventually be required. We describe in this paper the initial state of the system, the various debugging processes that were used, and our plans to further improve scalability.

Calcutt, J. [Brookhaven Natl. Lab.]

L'Arlesienne de ROOT

Over many years, ROOT users have repeatedly stumbled over—and loudly rediscovered—the infamous 1 GB limit on individual I/O operations, a constraint that somehow survived long past the era when anyone thought a gigabyte was “a lot.” As experiments embraced ever-larger objects and collections, this limit became an increasingly unavoidable rite of passage. This contribution recounts the sustained, multi-year quest by ROOT I/O developers to finally retire this relic, navigating a maze of legacy APIs, memory-management assumptions, and integer boundaries that seemed determined to preserve the status quo. We describe how internal interfaces were carefully modernized to introduce fully 64-bit–capable code paths without breaking the mountains of existing user code that would definitely have noticed. With the limit now lifted, ROOT can finally handle multi-gigabyte objects in a single read or write operation, even when splitting them into an RNTuple is not an option (we’re looking at you, large RooWorkspaces and giant histograms), liberating users from yet another “fun” debugging adventure and clearing the way for the massive analyses of the HL-LHC and beyond.

Canal, Philippe G. [Fermilab] (ORCID:0000000277487

L'Arlesienne de ROOT

Over many years, ROOT users have repeatedly stumbled over—and loudly rediscovered—the infamous 1 GB limit on individual I/O operations, a constraint that somehow survived long past the era when anyone thought a gigabyte was “a lot.” As experiments embraced ever-larger objects and collections, this limit became an increasingly unavoidable rite of passage. This contribution recounts the sustained, multi-year quest by ROOT I/O developers to finally retire this relic, navigating a maze of legacy APIs, memory-management assumptions, and integer boundaries that seemed determined to preserve the status quo. We describe how internal interfaces were carefully modernized to introduce fully 64-bit–capable code paths without breaking the mountains of existing user code that would definitely have noticed. With the limit now lifted, ROOT can finally handle multi-gigabyte objects in a single read or write operation, even when splitting them into an RNTuple is not an option (we’re looking at you, large RooWorkspaces and giant histograms), liberating users from yet another “fun” debugging adventure and clearing the way for the massive analyses of the HL-LHC and beyond.

Canal, Philippe G. [Fermilab] (ORCID:0000000277487

Knowledge Graph for End-to-End Traceability of an Integrated Human-Earth System Model

Integrated human-Earth system models inform energy-water-land system dynamics and policies, yet their results are difficult to trace through input-data, model structure, scenario configurations, and solved outputs. Because this information is siloed across disconnected artifacts, process-based IAMs have historically lacked a unified, queryable representation. Such lack of traceability prevents researchers from systematically isolating the multi-sector drivers of complex outcomes (such as tracing water-scarcity results back to distant energy-system dynamics) or conducting holistic uncertainty attribution across hundreds of interacting parameters. To address this concern, our work documents the software engineering process of a knowledge graph that unifies these four layers for the Global Change Analysis Model (GCAM-USA_Reference scenario, GCAM v9.1). The graph was built as a relational property graph in DuckDB from the run’s own artifacts: the input-preparation dependency map (gcamdata chunk map), the model’s XML input files, the run configuration, and the results database (BaseX), successfully mapping the model’s declared structure. The resulting graph comprises 204,321 nodes and 1,687,814 edges across 16 node types and 15 edge types, with approximately 16.3 million time-series values stored separately to maintain structural efficiency. To ensure representation fidelity, every edge carries an epistemic-status annotation recording the warrant for the relationship (structural, provenance, dependency, or model-derived), and a machine-readable provenance ledger classifying the origin of every schema element. Evaluation against a fixed five-benchmark suite with locked baselines reports zero structural orphans, zero dangling edge endpoints, and 100% of output-producing technologies traceable to raw input files. Two interactive interfaces present the graph, including a serverless browser application built on DuckDB-Wasm. By establishing the first end-to-end provenance framework for an IAM, this work enables researchers and scientists to systematically audit complex policy scenarios, debug model structures, and trace policy-relevant outputs to their data origins in real time.

Artifical Intelligence

Idiomatic Correctness-Checking via Julienne in Fortran 2023

This paper presents a unified approach to unit testing and runtime assertion checking using Fortran 2023. The paper describes the support for our approach in the Julienne framework. Julienne leverages recent Fortran standards to implement object-oriented design patterns, support testing parallel programs, and implement functional programming patterns in order to craft idioms inspired by natural-language expressions. The presented idioms employ novel operators to write expressions that evaluate to a test-diagnosis object encapsulating two components: (1) the test outcome or assertion outcome and (2) an automatically generated diagnostic string. Two other novel aspects of the approach include (1) the ability to enforce assertions inside pure procedures and (2) the ability to output rich diagnostic information inside pure procedures during error termination when assertions fail. The latter capability mitigates against a reason that Fortran programmers commonly cite for not writing pure procedures: difficulty obtaining useful program output inside pure procedures when debugging code. This paper demonstrates how the adoption of the proposed idioms leads naturally to a unifying theme across two otherwise disparate technologies: unit testing and runtime assertion checking. Finally, this paper describes the usage of the Julienne testing framework for writing unit tests and assertions in the Matcha high-performance computing application and the Fiats deep learning library.

Rouson, Damian

Using Containers to Speed Up Development, to Run Integration Tests and to Teach About Distributed Systems

GlideinWMS is a workload manager provisioning resources for many experiments, including CMS and DUNE. The software is distributed both as native packages and specialized production containers. Following an approach used in other communities like web development, we built our workspaces, system-like containers to ease development and testing. Developers can change the source tree or check out a different branch and quickly reconfigure the services to see the effect of their changes. In this paper, we will talk about what differentiates workspaces from other containers. We will describe our base system, composed of three containers: a one-node cluster including a compute element and a batch system, a GlideinWMS Factory controlling pilot jobs, and a scheduler and Frontend to submit jobs and provision resources. Additional containers can be used for optional components. This system can easily run on a laptop, and we will share our evaluation of different container runtimes, with an eye for ease of use and performance. Finally, we will talk about our experience as developers and with students. The GlideinWMS workspaces are easily integrated with IDEs like VS Code, simplifying debugging and allowing development and testing of the system even when offline. They simplified the training and onboarding of new team members and summer interns. And they were useful in workshops where students could have first-hand experience with the mechanisms and components that, in production, run millions of jobs.

Mambelli, Marco [Fermilab] (ORCID:0000000294892681

NLR HPC Kestrel Jobs Data

Overview: Anonymized job-level records from the Kestrel HPC system at the National Laboratory of the Rockies (NLR). Each record represents a Slurm batch job with scheduling metadata, resource requests, utilization, energy estimates, and efficiency metrics. Sensitive fields (user, account, job name, submit line, working directory, submit script, and job type) are replaced with 7-character cryptographic hashes. System & Timeframe: Kestrel is located at the NLR campus. Standard compute nodes have 104 cores and 256 GB RAM; bigmem nodes have 2,000 GB. GPU nodes (gpu-h100 partition) use NVIDIA H100 GPUs. Data covers jobs submitted August 2023 through December 2025. Funding provided by the U.S. Department of Energy, EERE. Files: esif.hpc.kestrel.job-anon.zip — Anonymized job records (Hive-partitioned Parquet) datacard.md — Full dataset documentation ~11 million rows, 50 variables. Readable with PyArrow, pandas, DuckDB, Apache Spark, or any Parquet-compatible tool. Data Collection: Jobs collected via sacct with timezone-aware export (SLURM_TIME_FORMAT="%Y-%m-%dT%H:%M:%S%z"), loaded into PostgreSQL. Calculated columns updated via database triggers and batch functions. All timestamps use timestamptz and correctly handle DST transitions. Preprocessing: Anonymization of name, user, account, submit_line, work_dir, submit_script, and job_type via 7-char hex hashes Derived columns: queue_wait, cpu_eff, max/min/avg_mem_eff, energy estimates Simplified job state mapping (e.g., "CANCELLED by 132357" → "CANCELLED") Boolean flags: python_job, reframe_job Temporal decomposition: year, month, day, day_of_week, hour, minute from submit_time Shared node tracking: shared_job_count, nodes_shared, jobs_shared Key Variables: Scheduling: job_id, partition, state_simple, submit_time, start_time, end_time, queue_wait Resources: nodes_req/used, processors_req/used, memory_req, wallclock_req/used, gpus_requested Efficiency: cpu_eff, max/min/avg_mem_eff Energy: cpu_energy_tdp_estimated_max/used_watt_hours, consumed_energy_raw_joules, consumed_energy_raw_watt_hours Sharing: shared_job_count, nodes_shared, jobs_shared Partitions: short, standard, debug, gpu-h100 Job States: CANCELLED, COMPLETED, FAILED, PENDING, RUNNING QoS Levels: normal, high Important Notes: Timestamps include timezone offsets; DST transitions are handled correctly, though adding intervals across DST boundaries requires offset adjustment shared_job_count reflects physical node co-residency, not use of the shared partition Job step records and raw Slurm JSONB fields are excluded Do not attempt to re-identify individuals from hashed fields

97 MATHEMATICS AND COMPUTING

Advancing \textit{otsdaq}: Enhancements for Usability, Accuracy, and Robustness

High-energy physics (HEP) experiments require data acquisition (DAQ) systems that can orchestrate complex detector operations, high data throughput, and responsive, real-time feedback to operators. Traditional DAQ stacks, which are often bespoke, command-line driven and highly specific, impose large learning curves on users. The Off-The-Shelf Data Acquisition (\textit{otsdaq}) framework was created to address these issues by offering a highly customizable and scalable browser-based ’desktop’ environment, in which experiment-specific control and monitoring applications can be easily deployed and integrated. Although the initial development of the \textit{otsdaq} software was aimed at the Fermilab Test Beam Facility, \textit{otsdaq} is now being leveraged for broader deployment, including the upcoming Mu2e experiment, where real-time monitoring of field-programmable gate array (FPGA)-based Data Transfer Controllers (DTCs), Clock and Fanout (CFO) boards, and several other subsystems are critical. We contribute a set of targeted improvements to \textit{otsdaq}: bitmap visualization functionality for configured data, improved and corrected delta-based DTC throughput metrics, version control (VC)-backed source navigation for console messages, custom navigation hooks to eliminate disruptive user interface glitches, and copy-to-clipboard support for macro execution history. These changes improve usability, reduce debugging time, and increase accuracy in performance data as Mu2e moves toward commissioning.

Mohammed, A. [Unlisted, US]

A Graphical User Interface for the Deep Underground Neutrino Experiment Robotic Test Stand

In preparation for DUNE, Fermilab along with six other institutions are testing cold electronics for quality control before components placed in the far detector. We test them by using a robotic arm that places these chips into sockets on a computer board that will test their functionality. Up until now, the chips have been tested using a command line script that drives a state machine to conduct tests step-by-step. In order to lower the skill barrier to conduct tests and to speed up the quality control process, I was tasked to create a graphical user interface that would allow users to use buttons, text boxes, and drop-down menus to input information and tell the testing state machine how to operate. I had to learn about the Python package Tkinter to start the process of widget placement. I further developed a pause feature unused in the previous command line script that would allow the user to shut down testing gracefully, bring the robotic arm to go back to ground state, and go forward or backward a step in the testing process. After completing the basic functionality of the GUI, I started testing production chips with the GUI to debug. Some issues were found, which required me to further develop parts of the inherited state machine code. The code for the GUI has now been pushed into the copy the DUNE/FD_CE git repository and will soon be merged with the official DUNE/FD_CE repository so that the other institutions testing DUNE cold electronics can use and expand upon it.

Gutierrez Villanueva, Jaziel [Fermilab]