SAPHIRE 8 Software Acceptance Test Plan
This document describe & report the overall SAPHIRE 8 Software acceptance test paln to offically release the SAPHIRE version 8 software to the NRC custoer for distribution.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
This document describe & report the overall SAPHIRE 8 Software acceptance test paln to offically release the SAPHIRE version 8 software to the NRC custoer for distribution.
The purpose of the Independent Verification and Validation (IV&V) role in the evaluation of the SAPHIRE 8 Software Acceptance Test Plan is to assess the approach to be taken for intended testing activities. The plan typically identifies the items to be tested, the requirements being tested, the testing to be performed, test schedules, personnel requirements, reporting requirements, evaluation criteria, and any risks requiring contingency planning. The IV&V team began this endeavor after the software engineering and software development of SAPHIRE had already been in production.
The software quality assurance oversight consists of updating and maintaining revision control of the SAPHIRE 8 quality assurance program documentation and of monitoring revision control of the SAPHIRE 8 source code. This report summarizes the oversight efforts through description of the revision control system (RCS) setup, operation and contents. Documents maintained under revision control include the Acceptance Test Plan (ATP), Configuration Management Plan, Quality Assurance Plan, Software Project Plan, Requirements Traceability Matrix (RTM), System Test Plan, SDP Interface Training Manual, and the SAPHIRE 8, "New Features and Capabilities Overview".
Summit, the latest flagship supercomputer deployed at Oak Ridge Leadership Computing Facility (OLCF), became the number one system in the Top500 list in June 2018 and remains in the top spot in the most recent edition of the list. An extensive acceptance test plan was developed to evaluate the unique features introduced in the Summit architecture and system software stack. The acceptance test also includes tests to ensure that the system is reliable, stable, and performant.
The software quality assurance oversight consists of updating and maintaining revision control of the System Analysis Program for Hands-on Integrated Reliability Evaluations (SAPHIRE) Version 8 quality assurance program documentation and of monitoring revision control of the SAPHIRE Version 8 source code. This report summarizes the oversight efforts through description of the revision control system (RCS) setup, operation and contents. Documents maintained under revision control include the Acceptance Test Plan (ATP), Configuration Management Plan, Quality Assurance Plan, Software Project Plan, Requirements Traceability Matrix (RTM), System Test Plan, Significance Determination Process (SDP) Interface Training Manual, and the SAPHIRE 8 “New Features and Capabilities Overview.” These documents are referred to and can be transmitted upon request of the Nuclear Regulatory Commission (NRC) Contracting Officer’s Representative (COR).
This test plan describes XRD sample characterization for uranium alloys. This test plan includes preferred sample preparation information, characterization method, instrument parameters, calibration method, analysis software, data-acceptance requirements, documentation/reporting requirements. Texture measurements will be taken based on relative peak shapes (e.g., peak height, area, integrated intensity, etc.), relative to the individual XRD pattern.
This document details the software requirements, test plan, and test results for VERAView. In this document, the test platform hardware and software are described. Section 2 provides a list of the tests run and their acceptability as a test report. Section 3 provides the step-by-step execution of these tests and their results.
The INL software developers use version control for both the formally released SAPHIRE versions, as well as for source code. For each formal release of the software, the developers perform an acceptance test: the software must pass a suite of automated tests prior to official release. Each official release of SAPHIRE is assigned a unique version identifier. The release is bundled into a standard installation package for easy and consistent set-up by individual users. Included in the release is a list of bug fixes and new features for the current release, as well as a history of those items for past releases. Each formal release of SAPHIRE will have passed an acceptance test. In addition to assignment of a unique version identifier for an official software release, each source code file is kept in a controlled library. Source code is a collection of all the computer instructions written by developers to create the finished product. The library is kept on a server, where back-ups are regularly made. This document describes the configuration management approach used as part of the SAPHIRE development.
This document describes the software test plan for VERAIn and provides appendices for the software requirements and software test report. In this document, the test platform hardware and software are described. Appendix A provides a list of the tests run and their acceptability as a test report, and Appendix B provides the list of low-level software requirements as a requirements traceability matrix.
Software verification and validation (V&V) hold an important role in the safety analysis for nuclear facilities. Codes and models utilized in calculations that demonstrate the safety basis as part of a license application or authorization must demonstrate an acceptable pedigree with regard to key critical characteristics that are comprised of the functional and performance requirements of the software. This document describes the methodology for verification and validation of the SAS4A/SASSYS-1 systems-level liquid-metal safety analysis software in support of DOE authorization of the Versatile Test Reactor (VTR). The overall goal of this V&V effort is to demonstrate the suitability of SAS4A/SASSYS-1 for evaluation of safety performance and characterization of safety margins in VTR. To that end, the V&V methodology will be used to demonstrate the adequacy of the component and physical models of SAS4A/SASSYS-1 for the corresponding VTR design elements. This methodology, which has been derived as a generalized approach to the U.S. NRC’s evaluation model development and assessment process (EMDAP) of Regulatory Guide 1.203, entails four high-level activities: software requirements identification, assessment base development, determination of acceptance, and gap closure. The methodology has been generalized to be suitable for both verification and validation of the software for the VTR project. Several high-level engineering analysis quality assurance measures as well as supporting V&V methodology documentation are also described in this document to help support demonstration of software acceptance. To support initial development of the SAS4A/SASSYS-1 V&V basis, existing verification test problems and systems benchmarks are also identified and briefly described in this document.
This document presents the software test plan for MPACT. The software requirements and software test report are provided in the appendices. The test platform hardware and software are described herein. Appendix A provides a list of the tests that were run and their acceptability as a test report. Appendix B provides the list of low-level software requirements as a requirements traceability matrix.
Purpose: To quantify the interplay effect in non-gated VMAT external beam delivery using realistic, clinically relevant 3D motion in an anthropomorphic lung phantom, and to determine if adding margins is sufficient to account for motion or if gating is required in all cases. Methods: A 4D motion stage was used to move a Virtual Water (VW) lung target containing a piece of radiochromic EBT3 film in an anthropomorphic chest phantom. A five-arc stereotactic body radiation therapy (SBRT) treatment was planned using a CT scan of the phantom in its stationary position, using planning parameters chosen to push the optimizer to achieve a highly-modulated plan. Two scenarios were delivered using a Varian TrueBeam: the first was delivered with the phantom and target both stationary and the second was delivered with the phantom stationary but the target moving in a realistic, irregular 3D elliptical pattern. A single piece of 4×4 cm{sup 2} film was used per fraction, located in the central coronal plane of the target. Film was calibrated on a 6 MV beam with dose values from 0.20 to 20 Gy. Results: Preliminary test films were analyzed in ImageJ and MatLab software. Dose maps were calculated on a central region of interest (ROI) delineated on both the motion-induced and stationary films. Both static and dynamic film dose maps agreed with planning values within acceptable uncertainty. Conclusion: Including a large number of arcs in a clinically realistic SBRT treatment could reduce the effect of motion interplay due to averaging. Because all clinics do not employ multiple arcs for SBRT lung treatments, it is still important to consider the effects of motion on treatment delivery. Further analysis on the treatment films, as well as a broader investigation other planning parameters, will be conducted.
Purpose: 2D-2D kV image guided radiation therapy (IGRT) credentialing evaluation for clinical trial qualification was historically qualitative through submitting screen captures of the fusion process. However, as quantitative DICOM 2D-2D and 2D-3D image registration tools are implemented in clinical practice for better precision, especially in centers that treat patients with protons, better IGRT credentialing techniques are needed. The aim of this work is to establish methodologies for quantitatively reviewing IGRT submissions based on DICOM 2D-2D and 2D-3D image registration and to test the methodologies in reviewing 2D-2D and 2D-3D IGRT submissions for RTOG/NRG Oncology clinical trials qualifications. Methods: DICOM 2D-2D and 2D-3D automated and manual image registration have been tested using the Harmony tool in MIM software. 2D kV orthogonal portal images are fused with the reference digital reconstructed radiographs (DRR) in the 2D-2D registration while the 2D portal images are fused with DICOM planning CT image in the 2D-3D registration. The Harmony tool allows alignment of the two images used in the registration process and also calculates the required shifts. Shifts calculated using MIM are compared with those submitted by institutions for IGRT credentialing. Reported shifts are considered to be acceptable if differences are less than 3mm. Results: Several tests have been performed on the 2D-2D and 2D-3D registration. The results indicated good agreement between submitted and calculated shifts. A workflow for reviewing these IGRT submissions has been developed and will eventually be used to review IGRT submissions. Conclusion: The IROC Philadelphia RTQA center has developed and tested a new workflow for reviewing DICOM 2D-2D and 2D-3D IGRT credentialing submissions made by different cancer clinical centers, especially proton centers. NRG Center for Innovation in Radiation Oncology (CIRO) and IROC RTQA center continue their collaborative efforts to enhance quality assurance services and to be consistently adaptive to the new advances in radiation therapy. This project was supported by NCI grants U10CA180868, U10CA180822, U24CA180803, U24CA12014 and PA CURE Grant.
This document serves as the System-Level Acceptance Test (SLAT) Plan for Site Name, City, Country. This test plan is to provide independent testing of the Radiation Detection System (RDS) installed at Site Name to verify that Customs has been delivered a fully-functioning system as required by all contractual commitments. The system includes all installed hardware and software components. The SLAT plan will verify that separate components are working individually and collectively from a system perspective.
The purpose of this document is to detail the software testing of FAST-1.0 through unit, integration, and assessment tests. More than 400 tests were designed to provide coverage of the requirements for FAST-1.0. These requirements were developed in the NRC Statement of Work (SOW) for NRC Agreement Number NRC-HQ-25-14-D-001 and are transcribed to this document as part of the Software Quality Assurance Plan for the FAST Code System, PNNL-28767. FAST 1.0 represents the merger of the FRAPCON and FRAPTRAN codes and as such FAST-1.0 performs steady state and transient fuel performance calculations described below. FAST-1.0 calculates the steady state response of light-water reactor fuel rods during long-term burnup. The code calculates temperature, pressure, and deformation of a fuel rod as functions of time-dependent fuel rod power and coolant boundary conditions. The phenomena modeled by the code include: 1) heat conduction through fuel and cladding to the coolant; 2) cladding elastic and plastic deformation; 3) fuel-cladding mechanical interaction; 4) fission gas release from the fuel and rod internal pressure; and 5) cladding oxidation. FAST is used to perform independent calculations for regulatory evaluations of fuel performance under normal operation and anticipated operational occurrences (AOOs). FAST calculates the temperature and deformation history of a fuel rod as a function of time-dependent fuel rod power and coolant boundary conditions. The phenomena modeled by FAST include: 1) heat conduction; 2) heat transfer from cladding to coolant; 3) elastic-plastic fuel and cladding deformation; 4) cladding oxidation; 5) fission gas release; and 6) fuel rod gas pressure. FAST code assessment, development and maintenance drive a significant portion of the NRC fuel research activities and the tools are used in a substantial number of regulatory products. Given the centrality of the FAST code to the effectiveness of fuel research, it is critical to assess, develop and maintain this tool.
The purpose of this study was to perform delivery quality assurance with ArcCHECK and 3DVH system (Sun Nuclear, FL) and to evaluate the suitability of this system for volumetric-modulated arc therapy (VMAT) (RapidArc [RA]) verification. This software calculates the delivered dose distributions in patients by perturbing the calculated dose using errors detected in fluence or planar dose measurements. The device is tested to correlate the gamma passing rate (%GP) and the composite dose predicted by 3DVH software. A total of 28 patients with prostate cancer who were treated with RA were analyzed. RA treatments were delivered to a diode array phantom (ArcCHECK), which was used to create a planned dose perturbation (PDP) file. The 3DVH analysis used the dose differences derived from comparing the measured dose with the treatment planning system (TPS)-calculated doses to perturb the initial TPS-calculated dose. The 3DVH then overlays the resultant dose on the patient's structures using the resultant “PDP” beams. Measured dose distributions were compared with the calculated ones using the gamma index (GI) method by applying the global (Van Dyk) normalization and acceptance criteria, i.e., 3%/3 mm. Paired differences tests were used to estimate statistical significance of the differences between the composite dose calculated using 3DVH and %GP. Also, statistical correlation by means of logistic regression analysis has been analyzed. Dose-volume histogram (DVH) analysis for patient plans revealed small differences between treatment plan calculations and 3DVH results for organ at risk (OAR), whereas planning target volume (PTV) of the measured plan was systematically higher than that predicted by the TPS. The t-test results between the planned and the estimated DVH values showed that mean values were incomparable (p < 0.05). The quality assurance (QA) gamma analysis 3%/3 mm showed that in all cases there were only weak-to-moderate correlations (Pearson r: 0.12 to 0.74). Moreover, clinically relevant differences increased with increasing QA passing rate, indicating that some of the largest dose differences occurred in the cases of high QA passing rates, which may be called “false negatives.” The clinical importance of any disagreement between the measured and the calculated dose is often difficult to interpret; however, beam errors (either in delivery or in TPS calculation) can affect the effectiveness of the patient dose. Further research is needed to determinate the role of a PDP-type algorithm to accurately estimate patient dose effect.
Purpose: To implement an end-to-end QA test of the radiation therapy imaging, planning and delivery process, aimed to assess the dosimetric agreement accuracy between planned and delivered treatment, in order to identify and correct possible sources of deviation. To establish an internal standard for machine commissioning acceptance. Methods: A test involving all steps of the radiation therapy: imaging, planning and delivery process was designed. The test includes analysis of point dose and planar dose distributions agreement between TPS calculated and measured dose. An ad hoc 16 cm diameter PMMA phantom was constructed with one central and four peripheral bores that can accommodate calibrated electron density inserts. Using Varian Eclipse 10.0 and Elekta XiO 4.50 planning systems, IMRT, RapidArc and 3DCRT with hard and dynamic wedges plans were planned on the phantom and tested. An Exradin A1SL chamber is used with a Keithley 35617EBS electrometer for point dose measurements in the phantom. 2D dose distributions were acquired using MapCheck and Varian aS1000 EPID.Gamma analysis was performed for evaluation of 2D dose distribution agreement using MapCheck software and Varian Portal Dosimetry Application.Varian high energy Clinacs Trilogy, 2100C/CD, 2000CR and low energy 6X/EX where tested.TPS-CT# vs. electron density table were checked for CT-scanners used. Results: Calculated point doses were accurate to 0.127% SD: 0.93%, 0.507% SD: 0.82%, 0.246% SD: 1.39% and 0.012% SD: 0.01% for LoX-3DCRT, HiX-3DCRT, IMRT and RapidArc plans respectively. Planar doses pass gamma 3% 3mm in all cases and 2% 2mm for VMAT plans. Conclusion: Implementation of a simple and reliable quality assurance tool was accomplished. The end-to-end proved efficient, showing excellent agreement between planned and delivered dose evidencing strong consistency of the whole process from imaging through planning to delivery. This test can be used as a first step in beam model acceptance for clinical use.
Purpose: The purpose of this project is to demonstrate that a non-expensive 3D-printer can be used to manufacture a 3D-bolus for external beam therapy. The printed bolus then can be modeled in our treatment planning system to ensure accurate dose delivery to the patient. Methods: We developed a simple method to manufacture a patient-specific custom 3Dbolus. The bolus is designed using Eclipse Treatment Planning System, contoured onto the patients CT images. The bolus file is exported from Eclipse to 3D-printer software, and then printed using a 3D printer. Various tests were completed to determine the properties of the printing material. Percent depth dose curves in this material were measured with electron and photon beams for comparison to other materials. In order to test the validity of the 3D printed bolus for treatment planning, a custom bolus was printed and tested on the Rando phantom using film for a dose plane comparison. We compared the dose plane measured on the film to the same dose plane exported from our treatment planning system using Film QA software. The gamma-dose distribution tool was used in our film analysis. Results: We compared point measurements throughout the dose plane and were able to achieve greater than 95% passing rate at 3% dose difference and 3 mm distance to agreement, which is our departments acceptable gamma pixel parameters. Conclusion: The printed 3D bolus has proven to be accurately modeled in our treatment planning system, it is more conformal to the patient surface and more durable than other bolus currently used (wax, superflab etc.). It is also more convenient and less costly than comparable bolus from milling machine companies.