Results for “software testing and debugging” — Page 3

NASA NTRS2003

Experimental Evaluation of Verification and Validation Tools on Martian Rover Software

To achieve its science objectives in deep space exploration, NASA has a need for science platform vehicles to autonomously make control decisions in a time frame that excludes intervention from Earth-based controllers. Round-trip light-time is one significant factor motivating autonomy capability, another factor is the need to reduce ground support operations cost. An unsolved problem potentially impeding the adoption of autonomy capability is the verification and validation of such software systems, which exhibit far more behaviors (and hence distinct execution paths in the software) than is typical in current deepspace platforms. Hence the need for a study to benchmark advanced Verification and Validation (V&V) tools on representative autonomy software. The objective of the study was to access the maturity of different technologies, to provide data indicative of potential synergies between them, and to identify gaps in the technologies with respect to the challenge of autonomy V&V. The study consisted of two parts: first, a set of relatively independent case studies of different tools on the same autonomy code, second a carefully controlled experiment with human participants on a subset of these technologies. This paper describes the second part of the study. Overall, nearly four hundred hours of data on human use of three different advanced V&V tools were accumulated, with a control group that used conventional testing methods. The experiment simulated four independent V&V teams debugging three successive versions of an executive controller for a Martian Rover. Defects were carefully seeded into the three versions based on a profile of defects from CVS logs that occurred in the actual development of the executive controller. The rest of the document is structured a s follows. In section 2 and 3, we respectively describe the tools used in the study and the rover software that was analyzed. In section 4 the methodology for the experiment is described; this includes the code preparation, seeding of defects, participant training and experimental setup. Next we give a qualitative overview of how the experiment went from the point of view of each technology; model checking (section 5), static analysis (section 6), runtime analysis (section 7) and testing (section 8). The find section gives some preliminary quantitative results on how the tools compared.

Brat, Guillaume↗

NASA NTRS1985

Designing a priority driven multi-frame rate flight executive

The Advanced Transport Operating System (ATOPS) project is a NASA operational research flight project which is concerned with upgrading the new generation of flight computers. In connection with this work, it becomes also necessary to reassess the adequacy of the flight executive and flight software. In the present discussion, attention is given to the ATOPS project upgrade, the operating flight system, the design criteria for the new flight executive, the implementation of the executive, a sample real-time execution, a real-time debug and test tool, and several experiences which could be useful to others considering a similar exercise.

Smith-Taylor, R.↗

NASA NTRS2016

Investigating the Simulink Auto-Coding Process

Model based program design is the most clear and direct way to develop algorithms and programs for interfacing with hardware. While coding "by hand" results in a more tailored product, the ever-growing size and complexity of modern-day applications can cause the project work load to quickly become unreasonable for one programmer. This has generally been addressed by splitting the product into separate modules to allow multiple developers to work in parallel on the same project, however this introduces new potentials for errors in the process. The fluidity, reliability and robustness of the code relies on the abilities of the programmers to communicate their methods to one another; furthermore, multiple programmers invites multiple potentially differing coding styles into the same product, which can cause a loss of readability or even module incompatibility. Fortunately, Mathworks has implemented an auto-coding feature that allows programmers to design their algorithms through the use of models and diagrams in the graphical programming environment Simulink, allowing the designer to visually determine what the hardware is to do. From here, the auto-coding feature handles converting the project into another programming language. This type of approach allows the designer to clearly see how the software will be directing the hardware without the need to try and interpret large amounts of code. In addition, it speeds up the programming process, minimizing the amount of man-hours spent on a single project, thus reducing the chance of human error as well as project turnover time. One such project that has benefited from the auto-coding procedure is Ramses, a portion of the GNC flight software on-board Orion that has been implemented primarily in Simulink. Currently, however, auto-coding Ramses into C++ requires 5 hours of code generation time. This causes issues if the tool ever needs to be debugged, as this code generation will need to occur with each edit to any part of the program; additionally, this is lost time that could be spent testing and analyzing the code. This is one of the more prominent issues with the auto-coding process, and while much information is available with regard to optimizing Simulink designs to produce efficient and reliable C++ code, not much research has been made public on how to reduce the code generation time. It is of interest to develop some insight as to what causes code generation times to be so significant, and determine if there are architecture guidelines or a desirable auto-coding configuration set to assist in streamlining this step of the design process for particular applications. To address the issue at hand, the Simulink coder was studied at a foundational level. For each different component type made available by the software, the features, auto-code generation time, and the format of the generated code were analyzed and documented. Tools were developed and documented to expedite these studies, particularly in the area of automating sequential builds to ensure accurate data was obtained. Next, the Ramses model was examined in an attempt to determine the composition and the types of technologies used in the model. This enabled the development of a model that uses similar technologies, but takes a fraction of the time to auto-code to reduce the turnaround time for experimentation. Lastly, the model was used to run a wide array of experiments and collect data to obtain knowledge about where to search for bottlenecks in the Ramses model. The resulting contributions of the overall effort consist of an experimental model for further investigation into the subject, as well as several automation tools to assist in analyzing the model, and a reference document offering insight to the auto-coding process, including documentation of the tools used in the model analysis, data illustrating some potential problem areas in the auto-coding process, and recommendations on areas or practices in the current Ramses model that should be further investigated. Several skills were required to be built up over the course of the internship project. First and foremost, my Simulink skills have improved drastically, as much of my experience had been modeling electronic circuits as opposed to software models. Furthermore, I am now comfortable working with the Simulink Auto-coder, a tool I had never used until this summer; this tool also tested my critical thinking and C++ knowledge as I had to interpret the C++ code it was generating and attempt to understand how the Simulink model affected the generated code. I had come into the internship with a solid understanding of Matlab code, but had done very little in using it to automate tasks, particularly Simulink tasks; along the same lines, I had rarely used shell script to automate and interface with programs, which I gained a fair amount of experience with this summer, including how to use regular expression. Lastly, soft-skills are an area everyone can continuously improve on; having never worked with NASA engineers, which to me seem to be a completely different breed than what I am used to (commercial electronic engineers), I learned to utilize the wealth of knowledge present at JSC. I wish I had come into the internship knowing exactly how helpful everyone in my branch would be, as I would have picked up on this sooner. I hope that having gained such a strong foundation in Simulink over this summer will open the opportunity to return to work on this project, or potentially other opportunities within the division. The idea of leaving a project I devoted ten weeks to is a hard one to cope with, so having the chance to pick up where I left off sounds appealing; alternatively, I am interested to see if there are any opening in the future that would allow me to work on a project that is more in-line with my research in estimation algorithms. Regardless, this summer has been a milestone in my professional career, and I hope this has started a long-term relationship between JSC and myself. I really enjoy the thought of building on my experience here over future summers while I work to complete my PhD at Missouri University of Science and Technology.

Gualdoni, Matthew J.↗

DOE OSTI2019

Diablo Test Suite: Code Coverage Analysis

An analysis of the Diablo source code covered by its Software Quality Assurance test suite was performed. Intel’s code coverage tool was used to determine the percentage of covered files, blocks, and functions. Excluding non-functional legacy files and single use developer-only files, the code coverage tool showed that 55% of blocks, the most applicable metric and closest to lines of code, are covered. For a large software code, 70-80% coverage is generally considered a reasonable target.A significant percentage of the uncovered code can be categorized as features that are not practical to test, duplicate/unused legacy code, and niche features only used by select individuals. Additionally, large segments of code dedicated to debugging and error handling are untested, as the test problems do not error out. The remaining untested code can be considered general purpose for use by the analyst/user community. These untested general purpose features are identified in the document and should be prioritized when new testing is added. Suggestions for additional test problems that would have the biggest impact on increasing the code coverage metric are presented. Finally, it is important to note that the single biggest way to increase the overall coverage metric is to delete old and unused code. While admittedly tedious to perform, future code development would also benefit markedly from this. Deleting unused code would likely push the coverage metric north of 60%, which is not too far off the 70-80% target. Overall, the current test suite covers the majority of commonly used features in Diablo, but there remains significant room for improvement. These areas for improvement have been identified.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

The Kokkos EcoSystem: Comprehensive Performance Portability For High Performance Computing

State of the art Engineering and Science codes have grown in complexity dramatically over the last two decades. As a consequence application teams have adopted more sophisticated development strategies, leveraging third party libraries, deploying comprehensive testing and using advanced debugging and profiling tools. In todays environment of diverse hardware platforms, these applications also desire performance portability - avoiding the need to duplicate work for various platforms - which makes it necessary that these tools and libraries also work across the various systems. The Kokkos EcoSystem provides that portable software stack. Based on the Kokkos Core Programming Model, the EcoSystem provides math libraries, interoperability capabilities with Python and Fortran, and Tools for analysing, debugging, and optimizing applications. In this paper we will provide an overview of the components, discuss some specific use cases, and highlight how co-designing these components enables a more developer friendly experience.

42 ENGINEERING↗

DOE OSTI2017

Build and Execute Environment

At exascale, the challenge becomes to develop applications that run at scale and use exascale platforms reliably, efficiently, and flexibly. Workflows become much more complex because they must seamlessly integrate simulation and data analytics. They must include down-sampling, post-processing, feature extraction, and visualization. Power and data transfer limitations require these analysis tasks to be run in-situ or in-transit. We expect successful workflows will comprise multiple linked simulations along with tens of analysis routines. Users will have limited development time at scale and, therefore, must have rich tools to develop, debug, test, and deploy applications. At this scale, successful workflows will compose linked computations from an assortment of reliable, well-defined computation elements, ones that can come and go as required, based on the needs of the workflow over time. We propose a novel framework that utilizes both virtual machines (VMs) and software containers to create a workflow system that establishes a uniform build and execution environment (BEE) beyond the capabilities of current systems. In this environment, applications will run reliably and repeatably across heterogeneous hardware and software. Containers, both commercial (Docker and Rocket) and open-source (LXC and LXD), define a runtime that isolates all software dependencies from the machine operating system. Workflows may contain multiple containers that run different operating systems, different software, and even different versions of the same software. We will run containers in open-source virtual machines (KVM) and emulators (QEMU) so that workflows run on any machine entirely in user-space. On this platform of containers and virtual machines, we will deliver workflow software that provides services, including repeatable execution, provenance, checkpointing, and future proofing. We will capture provenance about how containers were launched and how they interact to annotate workflows for repeatable and partial re-execution. We will coordinate the physical snapshots of virtual machines with parallel programming constructs, such as barriers, to automate checkpoint and restart. We will also integrate with HPC-specific container runtimes to gain access to accelerators and other specialized hardware to preserve native performance. Containers will link development to continuous integration. When application developers check code in, it will automatically be tested on a suite of different software and hardware architectures.

Guan, Qiang↗

DOE OSTI2017

Build and Execute Environment

At exascale, the challenge becomes to develop applications that run at scale and use exascale platforms reliably, efficiently, and flexibly. Workflows become much more complex because they must seamlessly integrate simulation and data analytics. They must include down-sampling, post-processing, feature extraction, and visualization. Power and data transfer limitations require these analysis tasks to be run in-situ or in-transit. We expect successful workflows will comprise multiple linked simulations along with tens of analysis routines. Users will have limited development time at scale and, therefore, must have rich tools to develop, debug, test, and deploy applications. At this scale, successful workflows will compose linked computations from an assortment of reliable, well-defined computation elements, ones that can come and go as required, based on the needs of the workflow over time. We propose a novel framework that utilizes both virtual machines (VMs) and software containers to create a workflow system that establishes a uniform build and execution environment (BEE) beyond the capabilities of current systems. In this environment, applications will run reliably and repeatably across heterogeneous hardware and software. Containers, both commercial (Docker and Rocket) and open-source (LXC and LXD), define a runtime that isolates all software dependencies from the machine operating system. Workflows may contain multiple containers that run different operating systems, different software, and even different versions of the same software. We will run containers in open-source virtual machines (KVM) and emulators (QEMU) so that workflows run on any machine entirely in user-space. On this platform of containers and virtual machines, we will deliver workflow software that provides services, including repeatable execution, provenance, checkpointing, and future proofing. We will capture provenance about how containers were launched and how they interact to annotate workflows for repeatable and partial re-execution. We will coordinate the physical snapshots of virtual machines with parallel programming constructs, such as barriers, to automate checkpoint and restart. We will also integrate with HPC-specific container runtimes to gain access to accelerators and other specialized hardware to preserve native performance. Containers will link development to continuous integration. When application developers check code in, it will automatically be tested on a suite of different software and hardware architectures.

Guan, Qiang↗

NASA NTRS2013

FPGA Flash Memory High Speed Data Acquisition

The purpose of this research is to design and implement a VHDL ONFI Controller module for a Modular Instrumentation System. The goal of the Modular Instrumentation System will be to have a low power device that will store data and send the data at a low speed to a processor. The benefit of such a system will give an advantage over other purchased binary IP due to the capability of allowing NASA to re-use and modify the memory controller module. To accomplish the performance criteria of a low power system, an in house auxiliary board (Flash/ADC board), FPGA development kit, debug board, and modular instrumentation board will be jointly used for the data acquisition. The Flash/ADC board contains four, 1 MSPS, input channel signals and an Open NAND Flash memory module with an analog to digital converter. The ADC, data bits, and control line signals from the board are sent to an Microsemi/Actel FPGA development kit for VHDL programming of the flash memory WRITE, READ, READ STATUS, ERASE, and RESET operation waveforms using Libero software. The debug board will be used for verification of the analog input signal and be able to communicate via serial interface with the module instrumentation. The scope of the new controller module was to find and develop an ONFI controller with the debug board layout designed and completed for manufacture. Successful flash memory operation waveform test routines were completed, simulated, and tested to work on the FPGA board. Through connection of the Flash/ADC board with the FPGA, it was found that the device specifications were not being meet with Vdd reaching half of its voltage. Further testing showed that it was the manufactured Flash/ADC board that contained a misalignment with the ONFI memory module traces. The errors proved to be too great to fix in the time limit set for the project.

Gonzalez, April↗

NASA NTRS2018

SpaceWire as a Cube-Sat Instrument Interface

SpaceWire is used in the control and data interface for an instrument on a pair of small satellites, one of which was launched in summer 2017. The instrument SpaceWire interface is implemented in a Field Programmable Gate Array as an instantiated core controlled by a LEON3FT CPU, which is also implemented as an instantiated core. The UT699 processor in the flight computer provides the spacecraft side’s SpaceWire interface. A simple message based protocol consisting of four message types was defined, based on existing SpaceWire standards. One was for passing commands to and responses from the instrument in the form of text strings similar to those from a system console where each line of text is passed in a SpaceWire message. Another was for passing spacecraft time to the instrument. The third was for transferring files using a subset of the Remote Memory Access Protocol (RMAP). The fourth was for retrieving science data from the instrument. A set of user application programming interface (API) routines provided an abstracted interface to both the serial console (used during debug) and the SpaceWire device interface. Early instrument development and testing was done with a set of utilities that controlled a Star-Dundee USB-SpaceWire brick providing a user interface similar to a serial console terminal emulator with the addition of file and data transfers. Later in the integration and test process, these utilities were integrated with the COSMOS ground systems software used for spacecraft control, providing a seamless transition from standalone instrument tests to benchtop flat-sat test and full spacecraft level tests.

Lux, James P.↗

NASA NTRS1991

Testing interconnected VLSI circuits in the Big Viterbi Decoder

The Big Viterbi Decoder (BVD) is a powerful error-correcting hardware device for the Deep Space Network (DSN), in support of the Galileo and Comet Rendezvous Asteroid Flyby (CRAF)/Cassini Missions. Recently, a prototype was completed and run successfully at 400,000 or more decoded bits per second. This prototype is a complex digital system whose core arithmetic unit consists of 256 identical very large scale integration (VLSI) gate-array chips, 16 on each of 16 identical boards which are connected through a 28-layer, printed-circuit backplane using 4416 wires. Special techniques were developed for debugging, testing, and locating faults inside individual chips, on boards, and within the entire decoder. The methods are based upon hierarchical structure in the decoder, and require that chips or boards be wired themselves as Viterbi decoders. The basic procedure consists of sending a small set of known, very noisy channel symbols through a decoder, and matching observables against values computed by a software simulation. Also, tests were devised for finding open and short-circuited wires which connect VLSI chips on the boards and through the backplane.

Onyszchuk, I. M.↗

NASA NTRS1987

Sensing and perception research for space telerobotics at JPL

PIFLEX is a pipelined-image processor that can perform elaborate computations whose exact nature is not fixed in the hardware, and that can handle multiple images. A wire-wrapped prototype PIFEX module has been produced and debugged, using a version of the convolver composed of three custom VLSI chips (plus the line buffers). A printed circuit layout is being designed for use with a single-chip convolver, leading to production of a PIFEX with about 120 modules. A high-level language for programming PIFEX has been designed, and a compiler will be written for it. The camera calibration software has been completed and tested. Two more terms in the camera model, for lens distortion, probably will be added later. The acquisition and tracking system has been designed and most of it has been coded in Pascal for the MicroVAX-II. The feature tracker, motion stereo module and stereo matcher have executed successfully. The model matcher is still under development, and coding has begun on the tracking initializer. The object tracker was running on a different computer from the VAX, and preliminary runs on real images have been performed there. Once all modules are working, optimization and integration will begin. Finally, when a sufficiently large PIFEX is available, appropriate parts of acquisition and tracking, including much of the feature tracker, will be programmed into PIFEX, thus increasing the speed and robustness of the system.

Gennery, Donald B.↗

NASA NTRS1981

Development techniques for generic software

In developing the first version of a generic implementation of X.25, Levels 2 and 3, three development techniques are examined: table driven finite state machine implementation, an integrated testing environment, and top down design. A finite state machine (FSM) is a powerful tool for both specifying and implementing protocols. This technique was used in the X.25 specification. A table driven implementation of the FSM was chosen to facilitate changes and simplify coding. The effect that this technique have on program size, speed of execution, coding time, and debugging time are investigated.

Hamilton, R. L.↗

NASA NTRS2004

Softc: An Operational Software Correlator

Softc has been used operationally for spacecraft navigation at JPL for over 2 years and will be JPL's Mark 5 correlator next year. Softc was written to be as close to an ideal correlator as possible, making approximations only below 10(exp -13) seconds. The program can correlate real USB, real LSB, or complex I/Q data sampled with 1, 2, 4. or 8-bit resolution, and was developed with strong debugging tools that made final debugging relatively quick. Softc's algorithms and program structure are fully documented. Timing tests on a recent Intel CPU show Softc processes 8 lags of 1-bit sampled data at 10 MSamples/sec, independent of sample rate.

Lowe, Stephen T.↗

NASA NTRS1976

Standardized development of computer software. Part 1: Methods

This work is a two-volume set on standards for modern software engineering methodology. This volume presents a tutorial and practical guide to the efficient development of reliable computer software, a unified and coordinated discipline for design, coding, testing, documentation, and project organization and management. The aim of the monograph is to provide formal disciplines for increasing the probability of securing software that is characterized by high degrees of initial correctness, readability, and maintainability, and to promote practices which aid in the consistent and orderly development of a total software system within schedule and budgetary constraints. These disciplines are set forth as a set of rules to be applied during software development to drastically reduce the time traditionally spent in debugging, to increase documentation quality, to foster understandability among those who must come in contact with it, and to facilitate operations and alterations of the program as requirements on the program environment change.

Tausworthe, R. C.↗

NASA NTRS2004

Development of Submillimeter SIS Mixers and Broadband HEMT Amplifiers

This is the final technical report for NASA grant NAG5-9493. entitled "Development of Submillimeter SIS Mixers and Broadband HEMT Amplifiers". The goal of this project was to develop and demonstrate a new generation of superconducting tunnel junction (SIS) receivers with extremely wide instantaneous (intermediate-frequency, or IF) bandwidths. of order 12 GHz. along with the wideband low-noise microwave HEMT (high electron mobility transistor) amplifiers which follow the SIS mixer. These wideband SIS/HEMT receivers would allow rapid submillimeter wavelength spectral line surveys to be carried out, for instance with the NASA airborne observatory SOFIA. and could potentially be useful for future submillimeter space missions such as SAFIR. In addition, there are potential NASA earth science applications. such as the monitoring of the distribution of chemical species in the stratosphere and troposphere using the limb-sounding technique. The overall goals of this project have been achieved: a broadband 200-300 SIS receiver was designed and constructed, and was demonstrated in the field through a test run at the Caltech Submillimeter Observatory on Mauna Kea. HI. The technical details are described in the appendices. which are primarily conference publications. but Appendix A also includes an unpublished summary of the latest results. The work on the SIS mixer design are described in the conference publications (appendices B and C). The "Supermix" software package that was developed at Caltech and used for the SIS design is also described in two conference papers, but has been substantially revised, debugged. and extended as part of the work completed for this grant. The Supermix package is made available to the community at no charge. The electromagnetic design of a radial waveguide probe similar to the one used in this work is described in a journal publication. Details of the novel fabrication procedure used for producing the SIS devices at JPL are also given in an upcoming journal article. Finally, details on the wideband HEMT amplifier design and noise characterization techniques are described in two publications.

Zmuidzinas, J.↗

NASA NTRS2008

Random Test Run Length and Effectiveness

A poorly understood but important factor in many applications of random testing is the selection of a maximum length for test runs. Given a limited time for testing, it is seldom clear whether executing a small number of long runs or a large number of short runs maximizes utility. It is generally expected that longer runs are more likely to expose failures -- which is certainly true with respect to runs shorter than the shortest failing trace. However, longer runs produce longer failing traces, requiring more effort from humans in debugging or more resources for automated minimization. In testing with feedback, increasing ranges for parameters may also cause the probability of failure to decrease in longer runs. We show that the choice of test length dramatically impacts the effectiveness of random testing, and that the patterns observed in simple models and predicted by analysis are useful in understanding effects observed.

software testing↗

DOE OSTI2023

Code Coverage Status of ARC Code-DIF3D

The Argonne Reactor Code (ARC) software system supports users in their fast reactor design goals by providing neutronic, thermal-hydraulic, and structural analysis capabilities. DIF3D plays a pivotal role in the ARC system as the primary homogenized assembly neutronic calculation methodology for fast reactor problems. Over its 40 years history, ARC software usage with DIF3D has been applied to numerous fast and thermal spectrum reactor analysis projects with good to excellent comparison against experiments. With continued improvement of computation resources, many of the geometry modeling capabilities in DIF3D that were primarily used in low order schemes are not really needed anymore. Today, the diffusion and transport capabilities of DIF3D-VARIANT are primarily used in the reactor design process with some scattered usage of DIF3D-FD and DIF3D-Nodal. In recent work, the DIF3D software verification was completed for DIF3D-FD and DIF3D-VARIANT on the geometry options used in the Versatile Test Reactor project. While we can be confident that these capabilities of DIF3D are well used and thus trusted, it does not demonstrate that all possible input options of DIF3D are actually working, but just those that were tested as part of VTR are and that they are correct. Thus, the purpose of the present work is to identify a set of test problems for DIF3D and assess the code coverage of DIF3D for those test problems. The goal is to document what parts of the existing DIF3D code are touched by the set of test problems and which are not. Because the verification work done on DIF3D-VARIANT and DIF3D-FD was focused on the most common uses of DIF3D for fast reactor analysis, the code coverage assessment of those capabilities is the highest priority. This will ensure that nothing is being missed by the existing verification test problems that DIF3D relies upon. The DIF3D-Nodal capability will also be inspected for code coverage as part of this work to further ensure that regular regression testing of DIF3D will trap any likely errors the end user might experience with the DIF3D software. The code coverage analysis of DIF3D was performed with the Code Coverage Tool of the Intel Fortran compiler which requires modifications to the compilation of DIF3D. The detailed coverage tables are given for each submodule of DIF3D separately, and for the submodules which are primarily developed for DIF3D, most of the source files could be at least partially touched. Most of the uncovered parts/files could be easily ignored, because they are either for error message and debugging output or obviously not needed by DIF3D. Out of the entire source codes of DIF3D, only a few uncovered modules deserve further investigation.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2025

Code Coverage Status of the ARC Code DIF3D

The Argonne Reactor Code (ARC) software system supports users in their fast reactor design goals by providing neutronic, thermal-hydraulic, and structural analysis capabilities. DIF3D plays a pivotal role in the ARC system as the primary homogenized assembly neutronic calculation methodology for fast reactor problems. Over its 40 years history, ARC software usage with DIF3D has been applied to numerous fast and thermal spectrum reactor analysis projects with good to excellent comparison against experiments. With continued improvement of computation resources, many of the geometry modeling capabilities in DIF3D that were primarily used in low order schemes are not really needed anymore. Today, the diffusion and transport capabilities of DIF3D-VARIANT are primarily used in the reactor design process with some scattered usage of DIF3D-FD and DIF3D-Nodal. In recent work, the DIF3D software verification was completed for DIF3D-FD and DIF3D-VARIANT on the geometry options used in the Versatile Test Reactor project. While we can be confident that these capabilities of DIF3D are well used and thus trusted, it does not demonstrate that all possible input options of DIF3D are actually working, but just those that were tested as part of VTR are and that they are correct. Thus, the purpose of the present work is to identify a set of test problems for DIF3D and assess the code coverage of DIF3D for those test problems. The goal is to document what parts of the existing DIF3D code are touched by the set of test problems and which are not. Because the verification work done on DIF3D-VARIANT and DIF3D-FD was focused on the most common uses of DIF3D for fast reactor analysis, the code coverage assessment of those capabilities is the highest priority. This will ensure that nothing is being missed by the existing verification test problems that DIF3D relies upon. The DIF3D-Nodal capability will also be inspected for code coverage as part of this work to further ensure that regular regression testing of DIF3D will trap any likely errors the end user might experience with the DIF3D software. The code coverage analysis of DIF3D was performed with the Code Coverage Tool of the Intel Fortran compiler which requires modifications to the compilation of DIF3D. The detailed coverage tables are given for each submodule of DIF3D separately, and for the submodules which are primarily developed for DIF3D, most of the source files could be at least partially touched. Most of the uncovered parts/files could be easily ignored, because they are either for error message and debugging output or obviously not needed by DIF3D. Out of the entire source codes of DIF3D, only a few uncovered modules deserve further investigation.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗