Search NASA⌕ Search

SEARCH · Search NASA

Results for “METRICS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Comparison of Several Global Mixing Performance Metrics for High-Speed Fuel Injectors

To experimentally assess and compare the mixing performance of high-speed fuel injectors for scramjet engines, quantitative global metrics are needed. The one-dimensional metric most commonly used to assess the degree of mixing completeness at a given downstream station is the mixing efficiency parameter. The experimental determination of the mixing efficiency parameter requires measurement of the spatial distributions of both the fuel mass fraction and the mass flux. Standard in-stream gas sampling techniques can be used to measure the fuel mass fraction distribution, however the mass flux distribution is not easily determined experimentally because it requires the measurement of three independent aerothermodynamic variables in addition to the mixture composition. For this reason, several metrics that can be calculated from the fuel distribution alone are commonly used to assess mixing performance. Because these other metrics do not provide a mass flux-weighted measure of the local degree of mixing completeness, they may not correlate well with the mixing efficiency parameter. Therefore, if the substitute metrics are to be used to compare the mixing performance of candidate fuel injector concepts, it is important to understand their relationships to the mixing efficiency parameter in a representative scramjet combustor flowfield. This work investigates the relationships between the mixing efficiency parameter and several substitute metrics that are able to be measured with the current experimental setup of the Enhanced Injection and Mixing Project at the NASA Langley Research Center for baseline strut and ramp injectors. The results of these comparisons have revealed that it is possible to glean different (i.e., incorrect) conclusions about which injector is the better mixer when the substitute mixing performance metrics are used instead of the mixing efficiency parameter, thereby highlighting the importance of mass flux-weighted mixing performance metrics.

Ground, Cody R.↗

Gravitational lensing of stars orbiting Sgr A* as a probe of the black hole metric in the Galactic center

We show that a possible astrophysical experiment, detection of lensed images of stars orbiting close to Sgr A*, can provide insight into the form of the metric around a black hole. We model Sgr A* as a black hole and add in a {proportional_to}(1/r{sup 2}) term to the Schwarzschild metric near the black hole. We then attempt to determine the effect of this extra term on the properties of the secondary images of the S stars in the Galactic center. When the (1/r{sup 2}) term is positive, this represents a Reissner-Nordstrom metric. We show that there is little observational difference between a Schwarzschild black hole and a Reissner-Nordstrom black hole, leading to the conclusion that secondary images may not be a useful probe of electrical charge in black holes. A negative value for the (1/r{sup 2}) term can enter through modified gravity scenarios. Although physically unlikely to apply in the case of a large black hole, the Randall-Sundrum II braneworld scenario admits a metric of this form, known as the tidal Reissner-Nordstrom metric. We use values of tidal charge (Q in (Q/r{sup 2})) ranging from -6.4M{sup 2} to 1.6M{sup 2}. A negative value of Q enhances the brightness of images at all times and creates an increase in brightness of up to 0.4 magnitudes for the secondary image of the star S2 at periapse. We show that for other stars with brighter secondary images and positions more aligned with the optic axis, using the tidal Reissner-Nordstrom metric with negative Q enhances the images as well. However, the effect is less pronounced. This effect is related to the increase in the size of the photon sphere in this spacetime and, therefore, should be noticeable in other metrics with a similar effect on the photon sphere. With the next generation of instruments and increased knowledge of radiation from Sgr A*, we may be able to use properties of secondary images to place constraints on the size of the (1/r{sup 2}) term. This knowledge will be useful in constraining any modified gravity theory that adds a similar term into the strong field near a black hole.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

The dynamics of metric-affine gravity

Highlights: > The role and the dynamics of the connection in metric-affine theories is explored. > The most general second order action does not lead to a dynamical connection. > Including higher order invariants excites new degrees of freedom in the connection. > f(R) actions are also discussed and shown to be a non- representative class. - Abstract: Metric-affine theories of gravity provide an interesting alternative to general relativity: in such an approach, the metric and the affine (not necessarily symmetric) connection are independent quantities. Furthermore, the action should include covariant derivatives of the matter fields, with the covariant derivative naturally defined using the independent connection. As a result, in metric-affine theories a direct coupling involving matter and connection is also present. The role and the dynamics of the connection in such theories is explored. We employ power counting in order to construct the action and search for the minimal requirements it should satisfy for the connection to be dynamical. We find that for the most general action containing lower order invariants of the curvature and the torsion the independent connection does not carry any dynamics. It actually reduces to the role of an auxiliary field and can be completely eliminated algebraically in favour of the metric and the matter field, introducing extra interactions with respect to general relativity. However, we also show that including higher order terms in the action radically changes this picture and excites new degrees of freedom in the connection, making it (or parts of it) dynamical. Constructing actions that constitute exceptions to this rule requires significant fine tuned and/or extra a priori constraints on the connection. We also consider f(R) actions as a particular example in order to show that they constitute a distinct class of metric-affine theories with special properties, and as such they cannot be used as representative toy theories to study the properties of metric-affine gravity.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Evaluation of performance metrics for the Wave Energy Prize converters tested at 1/20 th scale

It is expected that wave energy technologies will play a future role in providing clean renewable energy and diversifying energy portfolios; however, they are still at an early stage of development compared to other renewables, with varying archetypes proposed. As technologies advance toward commercialization, benchmarking is needed to quantify performance and costs. In this review, experimental datasets of Wave Energy Converter (WEC) devices tested in the final stage of the Wave Energy Prize (WEPrize) are compared and ranked using performance metrics found in the literature and those developed as WEPrize judging metrics at both U.S. and European representative wave climates. Because the WEPrize devices were tested under a set of identical sea states, which ranged from typical operating conditions to extreme storm events, consistent datasets were produced to facilitate comparison. This allows for a rare addition to the open literature on device performance trends. In addition, a reevaluation of trends established in previous power performance benchmarking studies is given. Trends found in previous studies were confirmed, except for the absorbed energy per characteristic mass metric, in which some of the WEPrize devices had higher values. Each of the metrics considered in this study has limitations due to the assumptions in simplifying the economic potential (e.g., power absorbed vs. a proxy to cost). In addition, each of these proxies is limited to the capital cost of a device, unlike the final metric used in the WEPrize, HPQ, which includes limited proxies of operational and capital expenditures, as well as array considerations. Recommendations are given for the use and potential modification of the metrics considered. Specifically, it is recommended that the ACE metric (from the WEPrize) be modified to more accurately include the other important system costs, such as the PTO and mooring, as well as installation, operation and maintenance costs.

16 TIDAL AND WAVE POWER↗

Optical metrics and projective equivalence

Trajectories of light rays in a static spacetime are described by unparametrized geodesics of the Riemannian optical metric associated with the Lorentzian spacetime metric. We investigate the uniqueness of this structure and demonstrate that two different observers, moving relative to one another, who both see the Universe as static may determine the geometry of the light rays differently. More specifically, we classify Lorentzian metrics admitting more than one hyper-surface orthogonal timelike Killing vector and analyze the projective equivalence of the resulting optical metrics. These metrics are shown to be projectively equivalent up to diffeomorphism if the static Killing vectors generate a group SL(2,R), but not projectively equivalent in general. We also consider the cosmological C metrics in Einstein-Maxwell theory and demonstrate that optical metrics corresponding to different values of the cosmological constant are projectively equivalent.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A comparison of metrics to evaluate the effects of hydro-facility passage stressors on fish

Hydropower is the most common form of renewable energy, and countries worldwide are considering expanding hydropower to new areas. One of the challenges of hydropower deployment is mitigation of the environmental impacts including water quality, habitat alterations, and ecosystem connectivity. For fish species that inhabit river systems with hydropower facilities, passage through the facility to access spawning and rearing habitats can be particularly challenging. Fish moving downstream through a hydro-facility can be exposed to a number of stressors (e.g., rapid decompression, shear forces, blade strike and collision, and turbulence), which can all affect fish survival in direct and indirect ways. Many studies have investigated the effects of hydro-turbine passage on fish; however, the comparability among studies is limited by variation in the metrics and biological endpoints used. Future studies investigating the effects of hydro-turbine passage should focus on using metrics and endpoints that are easily comparable. This review summarizes four categories of metrics that are used in fisheries research and have application to hydro-turbine passage (i.e., mortality, injury, molecular metrics, behavior) and evaluates them based on several criteria (i.e., resources needed, invasiveness, comparability among stressors and species, and diagnostic properties). Additionally, these comparisons are put into context of study setting (i.e., laboratory vs. field). Overall, injury and molecular metrics are ideal for studies in which there is a need to understand the mechanisms of effect, whereas behavior and mortality metrics provide information on the whole body response of the fish. The study setting strongly influences the comparability among studies. In laboratory-based studies, stressors can be controlled by both type and magnitude, allowing for easy comparisons among studies. In contrast, field studies expose fish to realistic passage environments but the comparability is limited. Based on these results, future studies, whether lab or field-based, should focus on metrics that relate to mortality for ease of comparison.

fish↗

Surveillance metrics sensitivity study.

In September of 2009, a Tri-Lab team was formed to develop a set of metrics relating to the NNSA nuclear weapon surveillance program. The purpose of the metrics was to develop a more quantitative and/or qualitative metric(s) describing the results of realized or non-realized surveillance activities on our confidence in reporting reliability and assessing the stockpile. As a part of this effort, a statistical sub-team investigated various techniques and developed a complementary set of statistical metrics that could serve as a foundation for characterizing aspects of meeting the surveillance program objectives. The metrics are a combination of tolerance limit calculations and power calculations, intending to answer level-of-confidence type questions with respect to the ability to detect certain undesirable behaviors (catastrophic defects, margin insufficiency defects, and deviations from a model). Note that the metrics are not intended to gauge product performance but instead the adequacy of surveillance. This report gives a short description of four metrics types that were explored and the results of a sensitivity study conducted to investigate their behavior for various inputs. The results of the sensitivity study can be used to set the risk parameters that specify the level of stockpile problem that the surveillance program should be addressing.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

SAPHIRE 8 Quality Assurance Software Metrics Report

The purpose of this review of software metrics is to examine the quality of the metrics gathered in the 2010 IV&V and to set an outline for results of updated metrics runs to be performed. We find from the review that the maintenance of accepted quality standards presented in the SAPHIRE 8 initial Independent Verification and Validation (IV&V) of April, 2010 is most easily achieved by continuing to utilize the tools used in that effort while adding a metric of bug tracking and resolution. Recommendations from the final IV&V were to continue periodic measurable metrics such as McCabe's complexity measure to ensure quality is maintained. The four software tools used to measure quality in the IV&V were CodeHealer, Coverage Validator, Memory Validator, Performance Validator, and Thread Validator. These are evaluated based on their capabilities. We attempted to run their latest revisions with the newer Delphi 2010 based SAPHIRE 8 code that has been developed and was successful with all of the Validator series of tools on small tests. Another recommendation from the IV&V was to incorporate a bug tracking and resolution metric. To improve our capability of producing this metric, we integrated our current web reporting system with the SpiraTest test management software purchased earlier this year to track requirements traceability.

97 MATHEMATICS AND COMPUTING↗

Surveillance Metrics Sensitivity Study

In September of 2009, a Tri-Lab team was formed to develop a set of metrics relating to the NNSA nuclear weapon surveillance program. The purpose of the metrics was to develop a more quantitative and/or qualitative metric(s) describing the results of realized or non-realized surveillance activities on our confidence in reporting reliability and assessing the stockpile. As a part of this effort, a statistical sub-team investigated various techniques and developed a complementary set of statistical metrics that could serve as a foundation for characterizing aspects of meeting the surveillance program objectives. The metrics are a combination of tolerance limit calculations and power calculations, intending to answer level-of-confidence type questions with respect to the ability to detect certain undesirable behaviors (catastrophic defects, margin insufficiency defects, and deviations from a model). Note that the metrics are not intended to gauge product performance but instead the adequacy of surveillance. This report gives a short description of four metrics types that were explored and the results of a sensitivity study conducted to investigate their behavior for various inputs. The results of the sensitivity study can be used to set the risk parameters that specify the level of stockpile problem that the surveillance program should be addressing.

97 MATHEMATICS AND COMPUTING↗

Self-benchmarking Guide for Data Centers: Metrics, Benchmarks, Actions

This guide describes energy efficiency metrics and benchmarks that can be used to track the performance of and identify potential opportunities to reduce energy use in data centers. This guide is primarily intended for personnel who have responsibility for managing energy use in existing data centers - including facilities managers, energy managers, and their engineering consultants. Additionally, data center designers may also use the metrics and benchmarks described in this guide for goal-setting in new construction or major renovation. This guide provides the following information: (1) A step-by-step outline of the benchmarking process. (2) A set of performance metrics for the whole building as well as individual systems. For each metric, the guide provides a definition, performance benchmarks, and potential actions that can be inferred from evaluating this metric. (3) A list and descriptions of the data required for computing the metrics. This guide is complemented by spreadsheet templates for data collection and for computing the benchmarking metrics. This guide builds on prior data center benchmarking studies supported by the California Energy Commission. Much of the benchmarking data are drawn from the LBNL data center benchmarking database that was developed from these studies. Additional benchmark data were obtained from engineering experts including facility designers and energy managers. This guide also builds on recent research supported by the U.S. Department of Energy's Save Energy Now program.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Self-benchmarking Guide for Cleanrooms: Metrics, Benchmarks, Actions

This guide describes energy efficiency metrics and benchmarks that can be used to track the performance of and identify potential opportunities to reduce energy use in laboratory buildings. This guide is primarily intended for personnel who have responsibility for managing energy use in existing laboratory facilities - including facilities managers, energy managers, and their engineering consultants. Additionally, laboratory planners and designers may also use the metrics and benchmarks described in this guide for goal-setting in new construction or major renovation. This guide provides the following information: (1) A step-by-step outline of the benchmarking process. (2) A set of performance metrics for the whole building as well as individual systems. For each metric, the guide provides a definition, performance benchmarks, and potential actions that can be inferred from evaluating this metric. (3) A list and descriptions of the data required for computing the metrics. This guide is complemented by spreadsheet templates for data collection and for computing the benchmarking metrics. This guide builds on prior research supported by the national Laboratories for the 21st Century (Labs21) program, supported by the U.S. Department of Energy and the U.S. Environmental Protection Agency. Much of the benchmarking data are drawn from the Labs21 benchmarking database and technical guides. Additional benchmark data were obtained from engineering experts including laboratory designers and energy managers.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Self-benchmarking Guide for Laboratory Buildings: Metrics, Benchmarks, Actions

This guide describes energy efficiency metrics and benchmarks that can be used to track the performance of and identify potential opportunities to reduce energy use in laboratory buildings. This guide is primarily intended for personnel who have responsibility for managing energy use in existing laboratory facilities - including facilities managers, energy managers, and their engineering consultants. Additionally, laboratory planners and designers may also use the metrics and benchmarks described in this guide for goal-setting in new construction or major renovation. This guide provides the following information: (1) A step-by-step outline of the benchmarking process. (2) A set of performance metrics for the whole building as well as individual systems. For each metric, the guide provides a definition, performance benchmarks, and potential actions that can be inferred from evaluating this metric. (3) A list and descriptions of the data required for computing the metrics. This guide is complemented by spreadsheet templates for data collection and for computing the benchmarking metrics. This guide builds on prior research supported by the national Laboratories for the 21st Century (Labs21) program, supported by the U.S. Department of Energy and the U.S. Environmental Protection Agency. Much of the benchmarking data are drawn from the Labs21 benchmarking database and technical guides. Additional benchmark data were obtained from engineering experts including laboratory designers and energy managers.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Minimal-Area Metrics on the Swiss Cross and Punctured Torus

The closed string theory minimal-area problem asks for the conformal metric of least area on a Riemann surface with the condition that all non-contractible closed curves have length greater than or equal to an arbitrary constant that can be set to one. Through every point in such a metric there is a geodesic that saturates the length condition, and the saturating geodesics in a given homotopy class form a band. The extremal metric is unknown when bands of geodesics cross, as it happens for surfaces of non-zero genus. Here, we use recently proposed convex programs to numerically find the minimal-area metric on the square torus with a square boundary, for various sizes of the boundary. For large enough boundary the problem is equivalent to the “Swiss cross” challenge posed by Strebel. We find that the metric is positively curved in the two-band region and flat in the single-band regions. For small boundary the metric develops a third band of geodesics wrapping around it, and has both regions of positive and of negative curvature. This surface can be completed to provide the minimal-area metric on a once-punctured torus, representing a closed-string tadpole diagram.

97 MATHEMATICS AND COMPUTING↗

Reliability metrics and their management implications for open pond algae cultivation

The prevalence of contaminating organisms in outdoor algae cultivation, with the often-associated dramatic crop failures, necessitates the need for metrics describing production system reliability. Standard metrics for algae cultivation reliability are critically needed to be able to map improvements in operational parameters, but do not currently exist. In this work, we present a set of standard metrics including mean time to failure (MTTF) and mean time between failures (MTBF) as the basis for the calculation of pond failure rate (FR) and reliability coefficient (RC). Metrics associated with the numbers of contaminating organisms such as abundance ratio (AR), prevalence of infection (PI), and mean intensity of infection (MII) are also relevant. These metrics, based on measured experimental values of outdoor pond performance during the Algae Testbed Public-Private Partnership (ATP3) Unified Field Studies (UFS), provide a basis for quantifying pond failure and provide insight into potential pond management and contaminant mitigation strategies. From these reliability metrics applied to this dataset, we are able to link operational parameters, such as harvest frequency and inoculum source, to pond reliability for different algae strains and seasons, and from an assessment of AR, provide contamination thresholds beyond which a culture may be unrecoverable. Ultimately, the implementation of widely-practiced and simple-to-calculate algae pond reliability metrics calculated from rapid and easy to collect data or observations will reduce risk and uncertainty in large-scale algae deployment and aid in the development of integrated pest management (IPM) strategies.

09 BIOMASS FUELS↗

Ecologically inspired metrics for transitioning to a sustainable and resilient circular economy with application to multilayer plastic films

Current Circular Economy (CE) frameworks applied to product chains exhibit notable shortcomings. These include neglecting the resilience and robustness of design, requiring detailed economic and environmental data for impact assessment, and relying on qualitative rather than quantitative metrics capturing certain CE design aspects. In the current contribution, we addressed these shortcomings by developing an Ecologically inspired (Eco-inspired) Framework using the mathematical foundations of Ecological Network Analysis (ENA). While ENA metrics have previously found application in designing circular economies, particularly in Industrial Symbiosis (IS) networks, our adaptation tailors these metrics for use in product-level CE, recognizing the inherent distinctions between product-level CE and IS. Our Eco-inspired Framework comprises three key categories to provide holistic and granular-level metrics for designing product-level CE. The first set of metrics assesses circularity and resource efficiency. The second set gauges network intensity and robustness as complementary indicators ensuring a CE is both sustainable and resilient. The third group of metrics evaluates enhancement potential of CE strategies through introducing quantitative metrics for measuring the degree of closed-loop strategies and average circularity level of a CE design. The three comprehensive set of indicators within the Eco-inspired Framework uniquely captures various facets of circular design, whether originating from technological innovations and recovery improvement at the end of life (EoL), shifts in human consumption patterns, alterations in product design, or changes in business models. The framework’s application is tested in designing a CE for multilayer Polyethylene-Polyamide (PE-PA) films. Using the Eco-inspired Framework, we identified the best strategy for designing a resilient and sustainable CE for PE-PA films. A diverse set of EoL strategies along with a reduction in product consumption can improve circularity and resilience by 650% and 255%, respectively, and mitigate greenhouse gas emissions by 90%. The framework minimizes trade-offs between sustainability and circularity goals and offers insights on how to enhance each strategy for achieving a resilient and sustainable CE for products.

54 ENVIRONMENTAL SCIENCES↗

A New Evaluation Metric for Demand Response-Driven Real-Time Price Prediction Towards Sustainable Manufacturing

Abstract The increasing industry energy demand highlights the urgency of demand response management, while the emerging smart manufacturing technologies pave the way for the implementation of real-time price (RTP)-based demand response management towards sustainable manufacturing. The demand response management requires scheduling of manufacturing systems based on RTP predictions, and thus the prediction quality can directly alter the effectiveness of demand response. However, since the general price prediction algorithms and prediction evaluation metrics are not specifically designed for RTP in demand response problems, a good RTP prediction obtained and evaluated by these algorithms and metrics may not be suitable for demand response scheduling. Therefore, in this study, the relationships between the effectiveness of demand response for manufacturing systems and evaluation results from six commonly used metrics are investigated. Meanwhile, a new metric called k-peak distance (KPD), considering the characteristics of the demand response problem, is proposed and compared with the other six metrics. Furthermore, an encoder-decoder long short-term memory recurrent neural network with KPD is proposed to provide better RTP prediction for manufacturing demand response problems. The case studies indicate that the proposed KPD metric shows a 1.8–3.6 times higher correlation with the demand response effectiveness compared to the other metrics. In addition, the production schedule based on the RTP prediction obtained from the proposed algorithm can improve the effectiveness of demand response by 23.4% on average.

Engineering↗

Evaluation of the microangiographic fluoroscope (MAF) using generalized system performance metrics

Purpose: The performance of a newly developed, high resolution, microangiographic fluoroscope (MAF) (35 {mu}m pixel pitch and 300 {mu}m thick CsI phosphor) was evaluated using a generalized linear system analysis and compared with that of a standard amorphous Si thin film transistor flat panel detector (FPD) (194 {mu}m pixel pitch and 600 {mu}m thick CsI phosphor). The linear system metrics such as modulation transfer function (MTF), noise power spectrum, and detection quantum efficiency (DQE) are commonly used to gauge the intrinsic detector performance in the detector plane. However, these linear system metrics do not provide information about the image receptor performance in a real system since they do not include the effects of other parameters such as focal spot distribution, scatter radiation, and geometric unsharpness, which may compromise detector performance characteristics. Use of generalized linear system metrics [generalized modulation transfer function (GMTF), generalized normalized noise power spectrum (GNNPS), and generalized detection quantum efficiency (GDQE)] that include these effects gives a more meaningful, complete, and appropriate evaluation of detector performance as part of the imaging system. Methods: A uniform head equivalent phantom was used to simulate realistic clinical parameters and x-ray spectra. The detector MTFs were measured using the slanted edge method and the focal spot MTFs were measured using a pinhole assembly. The scatter MTF was simulated and the scatter fraction was measured for a head-equivalent phantom. The generalized system metrics were calculated for different combinations of three choices of focal spots and three different magnifications with two different air-gaps. The performance of the MAF was also illustrated using stent images obtained with different focal spots under similar conditions. Results: Results for the generalized metrics provide a quantitative description of the performance of the imaging system for both detectors. This generalized analysis demonstrated that both detectors have similar imaging capabilities at lower spatial frequencies, but that the MAF has superior performance over the FPD at higher frequencies even when considering focal spot blurring and scatter. Conclusions: This generalized performance analysis demonstrates the significance of focal spot size, magnification, and scatter on the system performance metrics (GMTF, GNNPS, and GDQE). Although the ideal detector performance characteristics of the MAF are not fully realized due to these other system factors, it still retains an advantage in DQE at high spatial frequencies over the FPD. Similar studies based on the generalized linear system metrics can serve as an efficient tool to evaluate total system capabilities under different realistic conditions to enable optimal design for specific imaging tasks.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Evaluating IMRT and VMAT dose accuracy: Practical examples of failure to detect systematic errors when applying a commonly used metric and action levels

Purpose: This study (1) examines a variety of real-world cases where systematic errors were not detected by widely accepted methods for IMRT/VMAT dosimetric accuracy evaluation, and (2) drills-down to identify failure modes and their corresponding means for detection, diagnosis, and mitigation. The primary goal of detailing these case studies is to explore different, more sensitive methods and metrics that could be used more effectively for evaluating accuracy of dose algorithms, delivery systems, and QA devices.Methods: The authors present seven real-world case studies representing a variety of combinations of the treatment planning system (TPS), linac, delivery modality, and systematic error type. These case studies are typical to what might be used as part of an IMRT or VMAT commissioning test suite, varying in complexity. Each case study is analyzed according to TG-119 instructions for gamma passing rates and action levels for per-beam and/or composite plan dosimetric QA. Then, each case study is analyzed in-depth with advanced diagnostic methods (dose profile examination, EPID-based measurements, dose difference pattern analysis, 3D measurement-guided dose reconstruction, and dose grid inspection) and more sensitive metrics (2% local normalization/2 mm DTA and estimated DVH comparisons).Results: For these case studies, the conventional 3%/3 mm gamma passing rates exceeded 99% for IMRT per-beam analyses and ranged from 93.9% to 100% for composite plan dose analysis, well above the TG-119 action levels of 90% and 88%, respectively. However, all cases had systematic errors that were detected only by using advanced diagnostic techniques and more sensitive metrics. The systematic errors caused variable but noteworthy impact, including estimated target dose coverage loss of up to 5.5% and local dose deviations up to 31.5%. Types of errors included TPS model settings, algorithm limitations, and modeling and alignment of QA phantoms in the TPS. Most of the errors were correctable after detection and diagnosis, and the uncorrectable errors provided useful information about system limitations, which is another key element of system commissioning.Conclusions: Many forms of relevant systematic errors can go undetected when the currently prevalent metrics for IMRT/VMAT commissioning are used. If alternative methods and metrics are used instead of (or in addition to) the conventional metrics, these errors are more likely to be detected, and only once they are detected can they be properly diagnosed and rooted out of the system. Removing systematic errors should be a goal not only of commissioning by the end users but also product validation by the manufacturers. For any systematic errors that cannot be removed, detecting and quantifying them is important as it will help the physicist understand the limits of the system and work with the manufacturer on improvements. In summary, IMRT and VMAT commissioning, along with product validation, would benefit from the retirement of the 3%/3 mm passing rates as a primary metric of performance, and the adoption instead of tighter tolerances, more diligent diagnostics, and more thorough analysis.

61 RADIATION PROTECTION AND DOSIMETRY↗