Search NASA⌕ Search

SEARCH · Search NASA

Results for “error impact analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

A non-intrusive optical (NIO) approach to characterize heliostats in utility-scale power tower plants: Sensitivity study

Power towers, a type of concentrating power tower technology, use a large number of heliostats to concentrate sunlight and produce renewable energy. Optical errors of heliostats can cause drastic losses in the efficiency of power-tower plants. Accurately measuring optical errors is crucial to assessing and improving plant performance. This analysis discusses an innovate non-intrusive optical (NIO) approach to measure slope, canting, and tracking errors by detecting distortions in the reflected tower structure in heliostat images. Further in this work a sensitivity study is carefully conducted to determine the uncertainty requirements to allow the method to calculate slope errors with an accuracy of 0.25 mrad. Measurement uncertainty sources include camera resolution and position uncertainty, tower position uncertainty, and number of collected images. Each uncertainty source is investigated to determine its impact on the accuracy of the slope-error calculation. A combination of theoretical results and experimental results obtained from data collected on a heliostat at Sandia National Laboratories is used to determine and validate uncertainty requirements. The analysis shows that a measurement uncertainty of 0.25 mrad can be realized by realistically controlling uncertainty sources when implementing the NIO method. It demonstrates the superior performance of NIO in performing in-situ optical characterization.

14 SOLAR ENERGY↗

Numerical error analysis of SOLPS-ITER simulations of EAST

Abstract Plasma edge simulations with codes like SOLPS-ITER are widely employed to interpret fusion experiments. However, numerical errors appearing in such simulations are rarely investigated, despite their potential large impact on simulation results. These errors consist of the statistical error and the bias, both resulting from the finite number of employed EIRENE Monte Carlo particles and incomplete convergence, and the discretization error due to the finite resolution of the computational grids. In this contribution, the resulting numerical errors on simulations of pure deuterium and neon seeded H-mode EAST discharges are examined. The statistical error can be kept small compared to other numerical error contributions by averaging the plasma profiles. This allows investigating the bias and discretization errors using Richardson extrapolation. It is shown that grid refinement and the number of employed Monte Carlo particles have the largest influence on the result, in agreement with similar studies of an ITER deuterium case. For the first time, numerical error bars on the entire simulated target profiles are determined showing that the largest numerical error is 17.9%, mainly due to the plasma grid discretization. On top, also numerical errors on simulated neutral pressures are investigated in detail, for which the statistical error is dominant. The analysis demonstrates which setup is needed to keep numerical errors limited: the SOLPS-ITER averaging procedure should be employed including enough EIRENE particles, and the involved grids should be sufficiently refined to reduce discretization errors.

Boeyaert, Dieter (ORCID:0000000309208660)↗

Sources of error in detonation calorimeters and error analysis for neat 2,4,6-triamino-1,3,5- trinitrobenzene (TATB)

Here, a calorimeter for measuring heats of detonation at Lawrence Livermore National Laboratory is described. A calibration precision of 0.2 % at the 95 % confidence interval (CI) is reported. Sources of uncertainty are discussed, including nonequivalent sources, which are those arising from differences between calibration and experimental tests. The systematic error due to nonequivalent sources is bounded to 0.19–0.22 % of the measured heat for a standard detonation test where the confinement material is gold. The recommendation is to correct for the systematic error by adding 0.19 % to the reported value and adding 0.03 % to the uncertainty. It is demonstrated that the precision of a detonation test is variable with testing duration because a source of uncertainty resides in the thermodynamic correction factor k6, which accounts for the contribution to heat by stirring; the generated power is additive and therefore highly impacted by total test time. The relative proportion also varies with the magnitude of heat release and sample size, adding variance to the weight of the error arising from the correction factor. A full error analysis based on the described sources of uncertainty is developed. The methodology is applied to a test series on neat 2,4,6-triamino-1,3,5- trinitrobenzene (TATB), demonstrating an ultimate precision of 0.7 % (single test) for materials testing and a relative standard deviation of 1.8 %.

47 OTHER INSTRUMENTATION↗

Benchmarking machine learning strategies for phase-field problems

Abstract We present a comprehensive benchmarking framework for evaluating machine-learning approaches applied to phase-field problems. This framework focuses on four key analysis areas crucial for assessing the performance of such approaches in a systematic and structured way. Firstly, interpolation tasks are examined to identify trends in prediction accuracy and accumulation of error over simulation time. Secondly, extrapolation tasks are also evaluated according to the same metrics. Thirdly, the relationship between model performance and data requirements is investigated to understand the impact on predictions and robustness of these approaches. Finally, systematic errors are analyzed to identify specific events or inadvertent rare events triggering high errors. Quantitative metrics evaluating the local and global description of the microstructure evolution, along with other scalar metrics representative of phase-field problems, are used across these four analysis areas. This benchmarking framework provides a path to evaluate the effectiveness and limitations of machine-learning strategies applied to phase-field problems, ultimately facilitating their practical application.

36 MATERIALS SCIENCE↗

Impact of uncertainties in the cosmological parameters on the measurement of primordial non-Gaussianity

We study the impact of cosmological parameters' uncertainties on estimates of the primordial non-Gaussianity (NG) parameter f{sub NL} in local and equilateral models of non-Gaussianity. We show that propagating these errors increases the f{sub NL} 1{sigma} uncertainty by a term {delta}f{sub NL}{sup local}/f{sub NL}{sup local}{approx_equal}16% for WMAP and {delta}f{sub NL}{sup local}/f{sub NL}{sup local}{approx_equal}5% for Planck in the local case, whereas for equilateral configurations the correction terms are {delta}f{sub NL}{sup equil}/f{sub NL}{sup equil}{approx_equal}14% and {delta}f{sub NL}{sup equil}/f{sub NL}{sup equil}{approx_equal}4%, respectively. If we assume for f{sub NL}{sup local} a central value <f{sub NL}{sup local}>{approx_equal}60, according to recent WMAP 5-year estimates, we obtain for Planck a final correction {delta}f{sub NL}{sup local}{approx_equal}3. Although not dramatic, this correction is at the level of the expected estimator uncertainty for Planck, and should then be taken into account when quoting the significance of an eventual future detection. In current estimates of f{sub NL} the cosmological parameters are held fixed at their best-fit values. We finally note that the impact of uncertainties in the cosmological parameters on the final f{sub NL} error bar would become totally negligible if the parameters were allowed to vary in the analysis and then be marginalized over.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Scaling Automatic Vector Data Alignment to Satellite Imagery

Given the tremendous volume of accessible Earth Observation (EO) data, there is a need to develop scalable Geospatial Artificial Intelligence (GeoAI) solutions for time-sensitive applications. Scalability in this context refers to rapidly processing large-scale EO data using high performance computing resources. Accurate mapping of the built environment from remote sensing (RS) imagery has been one of the crucial components in GeoAI workflows for a wide spectrum of humanitarian applications. Derived vector data of built environment is often leveraged for disaster preparedness and response activities. However, factors such as differences in ortho-rectification, atmospheric conditions and human error, results in spatial misalignment between vector data and the timely available RS imagery. Model training for downstream tasks such as object detection, change analysis, etc., is negatively impacted due to such spatial misalignment. Although there has been progress towards automatic alignment of vector data, the lack of scalability remains an open research challenge. This paper proposes to leverage parallel computing to optimize an automatic vector data alignment workflow. It further employs CPU-level multi-core parallelism for improving the performance of the workflow for scalable built environment mapping. We report observations and discuss findings from the preliminary experiments performed on the Summit Supercomputer.

Potnis, Abhishek↗

Enhancing ZFP: A Statistical Approach to Understanding and Reducing Error Bias in a Lossy Floating-Point Compression Algorithm

The amount of data generated and gathered in scientific simulations and data collection applications is continuously growing, putting mounting pressure on storage and bandwidth concerns. A means of reducing such issues is data compression; but, lossless data compression is typically ineffective when applied to floating-point data. Thus, users tend to apply a lossy data compressor, which allows for small deviations from the original data. It is essential to understand how the error from lossy compression impacts the accuracy of the data analytics. Thus, we must analyze not only the compression properties but the error as well. In this paper, we provide a statistical analysis of the error caused by ZFP compression, a state-of-the-art, lossy compression algorithm explicitly designed for floating-point data. We show that the error is indeed biased and propose simple modifications to the algorithm to neutralize the bias and further reduce the resulting error.

97 MATHEMATICS AND COMPUTING↗

Quality Assurance With Plan Veto: Reincarnation of a Record and Verify System and Its Potential Value

Purpose: To quantify the potential impact of the Integrating the Healthcare Enterprise–Radiation Oncology Quality Assurance with Plan Veto (QAPV) on patient safety of external beam radiation therapy (RT) operations. Methods and Materials: An institutional database of events (errors and near-misses) was used to evaluate the ability of QAPV to prevent clinically observed events. We analyzed reported events that were related to Digital Imaging and Communications in Medicine RT plan parameter inconsistencies between the intended treatment (on the treatment planning system) and the delivered treatment (on the treatment machine). Critical Digital Imaging and Communications in Medicine RT plan parameters were identified. Each event was scored for importance using the Failure Mode and Effects Analysis methodology. Potential error occurrence (frequency) was derived according to the collected event data, along with the potential event severity, and the probability of detection with and without the theoretical implementation of the QAPV plan comparison check. Failure Mode and Effects Analysis Risk Priority Numbers (RPNs) with and without QAPV were compared to quantify the potential benefit of clinical implementation of QAPV. Results: The implementation of QAPV could reduce the RPN values for 15 of 22 (71%) of evaluated parameters, with an overall average reduction in RPN of 68 (range, 0-216). For the 6 high-risk parameters (>200), the average reduction in RPN value was 163 (range, 108-216). The RPN value reduction for the intermediate-risk (200 > RPN > 100) parameters was (0-140). With QAPV, the largest RPN value for “Beam Meterset” was reduced from 324 to 108. The maximum reduction in RPN value was for Beam Meterset (216, 66.7%), whereas the maximum percentage reduction was for Cumulative Meterset Weight (80, 88.9%). Conclusion: This analysis quantifies the value of the Integrating the Healthcare Enterprise–Radiation Oncology QAPV implementation in clinical workflow. We demonstrate that although QAPV does not provide a comprehensive solution for error prevention in RT, it can have a significant impact on a subset of the most severe clinically observed events.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Assimilation of X-band Phased-Array Radar Data with EnKF for the Analysis and Warning Forecast of a Tornadic storm

The impact of assimilating China's operational X-band Phased-Array radar's (X-PAR) data on the analysis and warning forecast of the vortex structure and intensity of the June 8, 2018 Foshan, Guangdong province, tornadic storm was investigated for the first time using an Ensemble Kalman Filter (EnKF) data assimilation system. Both radar radial velocity (Vr) and reflectivity (Z) from two S-band operational radars and one X-PAR were assimilated. Deterministic forecasts were launched every 6 min from 05:42 UTC (20 min before the tornado touched down) to 06:00 UTC from the EnKF mean analysis field. Five experiments were conducted to examine the added capability of Z assimilation of the EnKF system, and to investigate the impact of assimilating X-PAR data on the analysis and prediction of the tornadic storm. Compared to the experiment without Z assimilation, the assimilation of Z reduced the analysis error and greatly reduced the forecast error of Z. Furthermore, the assimilation of X-PAR data greatly improved the vortex structure of the tornadic storm at low levels, and improved the intensity of the rear inflow of the tornadic storm, especially with a higher assimilation frequency. Compared to the experiments without X-PAR data assimilation, assimilating X-PAR data improved the predictability of tornadic storm.

54 ENVIRONMENTAL SCIENCES↗

Misclassification of causes of death among a small all-autopsied group of former nuclear workers: Death certificates vs. autopsy reports

The U.S. Transuranium and Uranium Registries performs autopsies on each of its deceased Registrants as a part of its mission to follow up occupationally-exposed individuals. This provides a unique opportunity to explore death certificate misclassification errors, and the factors that influence them, among this small population of former nuclear workers. Underlying causes of death from death certificates and autopsy reports were coded using the 10 th revision of the International Classification of Diseases (ICD-10). These codes were then used to quantify misclassification rates among 268 individuals for whom both full autopsy reports and death certificates with legible underlying causes of death were available. When underlying causes of death were compared between death certificates and autopsy reports, death certificates correctly identified the underlying cause of death’s ICD-10 disease chapter in 74.6% of cases. The remaining 25.4% of misclassified cases resulted in over-classification rates that ranged from 1.2% for external causes of mortality to 12.2% for circulatory disease, and under-classification rates that ranged from 7.7% for external causes of mortality to 47.4% for respiratory disease. Neoplasms had generally lower misclassification rates with 4.3% over-classification and 13.3% under-classification. A logistic regression revealed that the odds of a match were 2.8 times higher when clinical history was mentioned on the autopsy report than when it was not. Similarly, the odds of a match were 3.4 times higher when death certificates were completed using autopsy findings than when autopsy findings were not used. This analysis excluded cases where it could not be determined if autopsy findings were used to complete death certificates. The findings of this study are useful to investigate the impact of death certificate misclassification errors on radiation risk estimates and, therefore, improve the reliability of epidemiological studies.

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗

WE-H-BRC-05: Catastrophic Error Metrics for Radiation Therapy

Purpose: Intuitive evaluation of complex radiotherapy treatments is impractical, while data transfer anomalies create the potential for catastrophic treatment delivery errors. Contrary to prevailing wisdom, logical scrutiny can be applied to patient-specific machine settings. Such tests can be automated, applied at the point of treatment delivery and can be dissociated from prior states of the treatment plan, potentially revealing errors introduced early in the process. Methods: Analytical metrics were formulated for conventional and intensity modulated RT (IMRT) treatments. These were designed to assess consistency between monitor unit settings, wedge values, prescription dose and leaf positioning (IMRT). Institutional metric averages for 218 clinical plans were stratified over multiple anatomical sites. Treatment delivery errors were simulated using a commercial treatment planning system and metric behavior assessed via receiver-operator-characteristic (ROC) analysis. A positive result was returned if the erred plan metric value exceeded a given number of standard deviations, e.g. 2. The finding was declared true positive if the dosimetric impact exceeded 25%. ROC curves were generated over a range of metric standard deviations. Results: Data for the conventional treatment metric indicated standard deviations of 3%, 12%, 11%, 8%, and 5 % for brain, pelvis, abdomen, lung and breast sites, respectively. Optimum error declaration thresholds yielded true positive rates (TPR) between 0.7 and 1, and false positive rates (FPR) between 0 and 0.2. Two proposed IMRT metrics possessed standard deviations of 23% and 37%. The superior metric returned TPR and FPR of 0.7 and 0.2, respectively, when both leaf position and MUs were modelled. Isolation to only leaf position errors yielded TPR and FPR values of 0.9 and 0.1. Conclusion: Logical tests can reveal treatment delivery errors and prevent large, catastrophic errors. Analytical metrics are able to identify errors in monitor units, wedging and leaf positions with favorable sensitivity and specificity. In part by Varian.

60 APPLIED LIFE SCIENCES↗

THREE-POINT CORRELATION FUNCTIONS OF SDSS GALAXIES: CONSTRAINING GALAXY-MASS BIAS

We constrain the linear and quadratic bias parameters from the configuration dependence of the three-point correlation function (3PCF) in both redshift and projected space, utilizing measurements of spectroscopic galaxies in the Sloan Digital Sky Survey Main Galaxy Sample. We show that bright galaxies (M{sub r} < -21.5) are biased tracers of mass, measured at a significance of 4.5{sigma} in redshift space and 2.5{sigma} in projected space by using a thorough error analysis in the quasi-linear regime (9-27 h{sup -1} Mpc). Measurements on a fainter galaxy sample are consistent with an unbiased model. We demonstrate that a linear bias model appears sufficient to explain the galaxy-mass bias of our samples, although a model using both linear and quadratic terms results in a better fit. In contrast, the bias values obtained from the linear model appear in better agreement with the data by inspection of the relative bias and yield implied values of {sigma}{sub 8} that are more consistent with current constraints. We investigate the covariance of the 3PCF, which itself is a measurement of galaxy clustering. We assess the accuracy of our error estimates by comparing results from mock galaxy catalogs to jackknife re-sampling methods. We identify significant differences in the structure of the covariance. However, the impact of these discrepancies appears to be mitigated by an eigenmode analysis that can account for the noisy, unresolved modes. Our joint analysis of both redshift space and projected measurements allows us to identify systematic effects affecting constraints from the 3PCF.

79 ASTRONOMY AND ASTROPHYSICS↗

Early Battery Performance Prediction for Mixed Use Charging Profiles Using Hierarchal Machine Learning

A key step limiting how fast batteries can be deployed is the time necessary to provide evaluation and validation of performance. Using data analysis approaches, such as machine learning, the validation process can be accelerated. However, questions on the validity of projecting models trained on limited data or simple cycling profiles, such as constant current cycling, to real-world scenarios with complex loads remains. Here, we present the ability to predict performance with less than 1.2% mean absolute percent error when trained on cells aged using complex electric vehicle discharge profiles, and either AC Level 2 charge or DC Fast charge profiles, using only the first 45 cycles, namely 5% of the total testing time. While error is low across the projections, this study also highlights that battery lifetime analysis using only cycling data may not extrapolate safely to certain real-world conditions due to the impact of calendar degradation.

25 ENERGY STORAGE↗

Analysis of the impact of broad absorption lines on quasar redshift measurements with synthetic observations

ABSTRACT Accurate quasar classifications and redshift measurements are increasingly important to precision cosmology experiments. Broad absorption line (BAL) features are present in 15–20 per cent of all quasars, and these features can introduce systematic redshift errors, and in extreme cases produce misclassifications. We quantitatively investigate the impact of BAL features on quasar classifications and redshift measurements with synthetic spectra that were designed to match observations by the Dark Energy Spectroscopic Instrument (DESI) survey. Over the course of 5 yr, DESI aims to measure spectra for 40 million galaxies and quasars, including nearly three million quasars. Our synthetic quasar spectra match the signal-to-noise ratio and redshift distributions of the first year of DESI observations, and include the same synthetic quasar spectra both with and without BAL features. We demonstrate that masking the locations of the BAL features decreases the redshift errors by about 1 per cent and reduces the number of catastrophic redshift errors by about 80 per cent. We conclude that identifying and masking BAL troughs should be a standard part of the redshift determination step for DESI and other large-scale spectroscopic surveys of quasars.

(galaxies:) quasars: absorption↗

Bias-Variance Trade-Off in Physics-Informed Neural Networks with Randomized Smoothing for High-Dimensional PDEs

Physics-Informed Neural Networks (PINNs) have triggered a paradigm shift in scientific computing, leveraging mesh-free properties and robust approximation capabilities. While proving effective for low-dimensional partial differential equations (PDEs), the computational cost of PINNs remains a hurdle in high-dimensional scenarios. This is particularly pronounced when computing high-order and high-dimensional derivatives in the physics-informed loss. Randomized Smoothing PINN (RS-PINN) introduces Gaussian noise for stochastic smoothing of the original neural net model, enabling the use of Monte Carlo methods for derivative approximation, which eliminates the need for costly automatic differentiation. Despite its computational efficiency, especially in the approximation of high-dimensional derivatives, RS-PINN introduces biases in both loss and gradients, negatively impacting convergence, especially when coupled with stochastic gradient descent (SGD) algorithms. We present a comprehensive analysis of biases in RS-PINN, attributing them to the nonlinearity of the Mean Squared Error (MSE) loss as well as the intrinsic nonlinearity of the PDE itself. We propose tailored bias correction techniques, delineating their application based on the order of PDE nonlinearity. The derivation of an unbiased RS-PINN allows for a detailed examination of its advantages and disadvantages compared to the biased version. Specifically, the biased version has a lower variance and runs faster than the unbiased version, but it is less accurate due to the bias. To optimize the bias-variance trade-off, we combine the two approaches in a hybrid method that balances the rapid convergence of the biased version with the high accuracy of the unbiased version. In addition to methodological contributions, we present an enhanced implementation of RS-PINN. Extensive experiments on diverse high-dimensional PDEs, including Fokker-Planck, Hamilton-Jacobi-Bellman (HJB), viscous Burgers’, Allen-Cahn, and Sine-Gordon equations, illustrate the bias-variance trade-off and highlight the effectiveness of the hybrid RS-PINN. Empirical guidelines are provided for selecting biased, unbiased, or hybrid versions, depending on the dimensionality and nonlinearity of the specific PDE problem.

97 MATHEMATICS AND COMPUTING↗

Ice storage model-predictive control in an office building with PV: scenario, error and sensitivity analysis

Thermal energy storage (TES) can enable more building-sited renewable electricity generation and lower utility bill costs for buildings owners and occupants, especially when there are high demand and variable time-of-use (TOU) charges. A model predictive control (MPC) strategy can offer additional savings over a schedule-based control with added complexity and reliance on forecasts. Here, this study examines savings for medium office buildings with chiller plants in three locations with building-installed solar photovoltaics (PV) to understand the impact of MPC. Control setpoints are fixed by a schedule-based control or optimized by nonlinear MPC. These control setpoints are actuated within EnergyPlus building models to simulate the utility cost of the chiller plant. NLP solutions can be unstable or unrealistic, but our results show that by regularizing the NLP, the solutions can be reasonably followed by the building model. MPC models make simplifications that lead to errors once the controller is participating in and changing the operation of the building. These errors average 9 % across the cases, showing that the most important parts of the system are represented. The no-thermal load costs are computed to show that the optimization can in some cases achieve both the minimum TOU and minimum monthly demand costs by demand management while reducing TOU energy costs by energy arbitrage. The MPC saves 35–66 % in the annual chiller plant operating costs, which is an additional savings above the schedule by 1–33 %. PV and TES are complementary and mostly independent, but a load with PV often results in better performance for the schedule. Our case study and sensitivity analysis show the importance of modeling and optimization for complex rates, but also the circumstances wherein a simpler strategy achieves the same performance with less potential for error.

14 SOLAR ENERGY↗

SU-F-T-14: Dosimetric Impacts of Various Uncertainties in Cervical Cancer HDR Brachytherapy: Are Conventional Point Doses Good Surrogates for 3D Dosimetry?

Purpose: In the context of evaluating dosimetric impacts of a variety of uncertainties involved in HDR Tandem-and-Ovoid treatment, to study the correlations between conventional point doses and 3D volumetric doses. Methods: For 5 cervical cancer patients treated with HDR T&O, 150 plans were retrospectively created to study dosimetric impacts of the following uncertainties: (1) inter-fractional applicator displacement between two treatment fractions within a single insertion by applying Fraction#1 plan to Fraction#2 CT; (2) positional dwell error simulated from −5mm to 5mm in 1mm steps; (3) simulated temporal dwell error of 0.05s, 0.1s, 0.5s, and 1s. The original plans were based on point dose prescription, from which the volume covered by the prescription dose was generated as the pseudo target volume to study the 3D target dose effect. OARs were contoured. The point and volumetric dose errors were calculated by taking the differences between original and simulated plans. The correlations between the point and volumetric dose errors were analyzed. Results: For the most clinically relevant positional dwell uncertainty of 1mm, temporal uncertainty of 0.05s, and inter-fractional applicator displacement within the same insertion, the mean target D90 and V100 deviation were within 1%. Among these uncertainties, the applicator displacement showed the largest potential target coverage impact (2.6% on D90) as well as the OAR dose impact (2.5% and 3.4% on bladder D2cc and rectum D2cc). The Spearman correlation analysis shows a correlation coefficient of 0.43 with a p-value of 0.11 between target D90 coverage and H point dose. Conclusion: With the most clinically relevant positional and temporal dwell uncertainties and patient interfractional applicator displacement within the same insertion, the dose error is within clinical acceptable range. The lack of correlation between H point and 3D volumetric dose errors is a motivator for the use of 3D treatment planning in cervical HDR brachytherapy.

60 APPLIED LIFE SCIENCES↗

Evaluating IMRT and VMAT dose accuracy: Practical examples of failure to detect systematic errors when applying a commonly used metric and action levels

Purpose: This study (1) examines a variety of real-world cases where systematic errors were not detected by widely accepted methods for IMRT/VMAT dosimetric accuracy evaluation, and (2) drills-down to identify failure modes and their corresponding means for detection, diagnosis, and mitigation. The primary goal of detailing these case studies is to explore different, more sensitive methods and metrics that could be used more effectively for evaluating accuracy of dose algorithms, delivery systems, and QA devices.Methods: The authors present seven real-world case studies representing a variety of combinations of the treatment planning system (TPS), linac, delivery modality, and systematic error type. These case studies are typical to what might be used as part of an IMRT or VMAT commissioning test suite, varying in complexity. Each case study is analyzed according to TG-119 instructions for gamma passing rates and action levels for per-beam and/or composite plan dosimetric QA. Then, each case study is analyzed in-depth with advanced diagnostic methods (dose profile examination, EPID-based measurements, dose difference pattern analysis, 3D measurement-guided dose reconstruction, and dose grid inspection) and more sensitive metrics (2% local normalization/2 mm DTA and estimated DVH comparisons).Results: For these case studies, the conventional 3%/3 mm gamma passing rates exceeded 99% for IMRT per-beam analyses and ranged from 93.9% to 100% for composite plan dose analysis, well above the TG-119 action levels of 90% and 88%, respectively. However, all cases had systematic errors that were detected only by using advanced diagnostic techniques and more sensitive metrics. The systematic errors caused variable but noteworthy impact, including estimated target dose coverage loss of up to 5.5% and local dose deviations up to 31.5%. Types of errors included TPS model settings, algorithm limitations, and modeling and alignment of QA phantoms in the TPS. Most of the errors were correctable after detection and diagnosis, and the uncorrectable errors provided useful information about system limitations, which is another key element of system commissioning.Conclusions: Many forms of relevant systematic errors can go undetected when the currently prevalent metrics for IMRT/VMAT commissioning are used. If alternative methods and metrics are used instead of (or in addition to) the conventional metrics, these errors are more likely to be detected, and only once they are detected can they be properly diagnosed and rooted out of the system. Removing systematic errors should be a goal not only of commissioning by the end users but also product validation by the manufacturers. For any systematic errors that cannot be removed, detecting and quantifying them is important as it will help the physicist understand the limits of the system and work with the manufacturer on improvements. In summary, IMRT and VMAT commissioning, along with product validation, would benefit from the retirement of the 3%/3 mm passing rates as a primary metric of performance, and the adoption instead of tighter tolerances, more diligent diagnostics, and more thorough analysis.

61 RADIATION PROTECTION AND DOSIMETRY↗