Search NASA⌕ Search

SEARCH · Search NASA

Results for “quality assessment”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

3D-equivariant graph neural networks for protein model quality assessment

Quality assessment (QA) of predicted protein tertiary structure models plays an important role in ranking and using them. With the recent development of deep learning end-to-end protein structure prediction techniques for generating highly confident tertiary structures for most proteins, it is important to explore corresponding QA strategies to evaluate and select the structural models predicted by them since these models have better quality and different properties than the models predicted by traditional tertiary structure prediction methods. We develop EnQA, a novel graph-based 3D-equivariant neural network method that is equivariant to rotation and translation of 3D objects to estimate the accuracy of protein structural models by leveraging the structural features acquired from the state-of-the-art tertiary structure prediction method—AlphaFold2. We train and test the method on both traditional model datasets (e.g. the datasets of the Critical Assessment of Techniques for Protein Structure Prediction) and a new dataset of high-quality structural models predicted only by AlphaFold2 for the proteins whose experimental structures were released recently. Our approach achieves state-of-the-art performance on protein structural models predicted by both traditional protein structure prediction methods and the latest end-to-end deep learning method—AlphaFold2. It performs even better than the model QA scores provided by AlphaFold2 itself. The results illustrate that the 3D-equivariant graph neural network is a promising approach to the evaluation of protein structural models. Integrating AlphaFold2 features with other complementary sequence and structural features is important for improving protein model QA.

59 BASIC BIOLOGICAL SCIENCES↗

Rapid Quality Assessment in Additive Manufacturing: New Procedure Accelerates Build Quality Assessment. White paper

Accelerated growth of the additive manufacturing (AM) industry in recent years is accompanied by a rising need for methods to quickly assess quality at-scale. Current practices for quality inspection include nondestructive test methods and destructive testing of witness coupons, which are artifacts built alongside the actual part. However, these methods can be costly and time-consuming. Recognizing this need, the Additive Manufacturing Center of Excellence (AM CoE) initiated a project led by its partner, Auburn University, to develop rapid testing procedure using asbuilt samples tested in torsion to quantitatively assess build quality. The presented work developed a rapid testing procedure using as-built samples tested in torsion to quantify small variances for assessing build quality.

42 ENGINEERING↗

Monitoring and data quality assessment of the ATLAS liquid argon calorimeter

The liquid argon calorimeter is a key component of the ATLAS detector installed at the CERN Large Hadron Collider. The primary purpose of this calorimeter is the measurement of electron and photon kinematic properties. It also provides a crucial input for measuring jets and missing transverse momentum. An advanced data monitoring procedure was designed to quickly identify issues that would affect detector performance and ensure that only the best quality data are used for physics analysis. This article presents the validation procedure developed during the 2011 and 2012 LHC data-taking periods, in which more than 98% of the proton-proton luminosity recorded by ATLAS at a centre-of-mass energy of 7-8 TeV had calorimeter data quality suitable for physics analysis.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Partially coherent wavefront propagation simulations: Mirror and monochromator crystal quality assessment

We have applied fully-and partially-coherent synchrotron radiation wavefront propagation simulations, implemented in the “Synchrotron Radiation Workshop” (SRW) computer code, to analyse the effects of imperfect mirrors and monochromator at the Coherent Hard X-ray beamline. This beamline is designed for X-ray Photon Correlation Spectroscopy, a technique that heavily relies on the partial coherence of the X-ray beam and benefits from a careful preservation of the X-ray wavefront. We present simulations and a comparison with the measured beam profile at the sample position, which show the impact of imperfect optics on the wavefront.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Quality Assessment of GPU Power Profiling Mechanisms

Accurate component-level power measurements are nowadays essential for the design and optimization of high-performance computing (HPC) systems and applications. Particularly, as more and more heterogeneous HPC systems are developed, the characterizations of GPU power profiles have become extremely crucial because, although GPUs provide exceptional performance, they do consume substantial amounts of power. Currently, there are various GPU power profiling mechanisms available; however, there is no standard way to assess the quality of such profiling schemes. To address this issue, in this paper, we develop an assessment methodology to rate the quality and performance of the profiling mechanism itself. Specifically, we present the assessments of four different GPU power profiling techniques: (i) Nvidia's NVML via Allinea MAP, (ii) Nvidia's NVML via direct reads, and (iii) Penguin Computing's PowerInsight (PI) via two vendor-provided drivers, and (iv) PowerInsight via Allinea MAP. In addition, we discuss the effects of moving-average filters to explain the slow variations of some of the measured power profiles. Based on our assessment, the GPU power profiling mechanism using PI device outperforms the other schemes by reliably measuring the ground-truth power profile generated by a GPU stress-test benchmark.

Sen, Satyabrata↗

Analytical methods for online data quality assessment

This chapter provides a comprehensive overview of the main steps for algorithmic sensor signal quality assessment, which can enhance the decision-making process for water resource recovery facility (WRRF) operation and optimization. It introduces the concept of redundancy as the basis for data quality assessment. It also explains the typical data processing pipeline, which consists of preliminary analysis, data pre-processing, and specific algorithmic approaches. Each of these processes is presented and discussed in three separate sections. Importantly, this chapter introduces the main approaches for data quality assessment, provides guidelines for selecting the most suitable one and the key performance indicators to evaluate them and explains how to collect metadata through such an algorithmic approach.

Aguado, Daniel↗

A Refined Method to Translate Solar Data Quality Assessment Flags to Estimated Measurement Uncertainty

Integrating solar resource uncertainties due to radiometer measurement performance and operational data quality assessment can provide improved estimates of economic bankability, system design performance, and compliance of solar energy conversion systems. Estimating radiometer measurement uncertainty is an established procedure consistent with recognized best practices and international guidelines. SERI QC is a robust solar data quality assessment software tool that has been in continuous use for more than three decades. This report, the fourth of six for the Data Quality and Uncertainty Integration Project, presents a refined algorithm description for software to translate solar resource data quality assessment results into estimated uncertainty values in a Solar Resource Operational Uncertainty Integrator (SROUI) application. This algorithm requires three-component solar irradiance measurements - global horizontal irradiance, direct normal irradiance, and diffuse horizontal irradiance - collected at 1- to 60-minute intervals, as described in the previous deliverables. The development of this report as Deliverable 6.4 was an iterative process that included reviews and feedback from the project team on initial drafts designed to refine how the new software could best support determining solar resource data uncertainty. The results of this effort will contribute to the final software system development by National Renewable Energy Laboratory staff.

14 SOLAR ENERGY↗

Soil bioassays as tools for sludge compost quality assessment

Composting is a waste management technology that is becoming more widespread as a response to the increasing production of sewage sludge and the pressure for its reuse in soil. In this study, different bioassays (plant germination, earthworm survival, biomass and reproduction, and collembolan survival and reproduction) were assessed for their usefulness in the compost quality assessment. Compost samples, from two different composting plants, were taken along the composting process, which were characterized and submitted to bioassays (plant germination and collembolan and earthworm performance). Results from our study indicate that the noxious effects of some of the compost samples observed in bioassays are related to the low organic matter stability of composts and the enhanced release of decomposition endproducts, with the exception of earthworms, which are favored. Plant germination and collembolan reproduction inhibition was generally associated with uncomposted sludge, while earthworm total biomass and reproduction were enhanced by these materials. On the other hand, earthworm and collembolan survival were unaffected by the degree of composting of the wastes. However, this pattern was clear in one of the composting procedures assessed, but less in the other, where the release of decomposition endproducts was lower due to its higher stability, indicating the sensitivity and usefulness of bioassays for the quality assessment of composts.

09 BIOMASS FUELS↗

MaNGA DynPop – I. Quality-assessed stellar dynamical modelling from integral-field spectroscopy of 10K nearby galaxies: a catalogue of masses, mass-to-light ratios, density profiles, and dark matter

ABSTRACT This is the first paper in our series on the combined analysis of the Dynamics and stellar Population (DynPop) for the MaNGA survey in the final SDSS Data Release 17 (DR17). Here, we present a catalogue of dynamically determined quantities for over 10 000 nearby galaxies based on integral-field stellar kinematics from the MaNGA survey. The dynamical properties are extracted using the axisymmetric Jeans Anisotropic Modelling (JAM) method, which was previously shown to be the most accurate for this kind of study. We assess systematic uncertainties using eight dynamical models with different assumptions. We use two orientations of the velocity ellipsoid: either cylindrically aligned JAMcyl or spherically aligned JAMsph. We also make four assumptions for the models’ dark versus luminous matter distributions: (1) mass-follows-light, (2) free NFW dark halo, (3) cosmologically constrained NFW halo, (4) generalized NFW dark halo, i.e. with free inner slope. In this catalogue, we provide the quantities related to the mass distributions (e.g. the density slopes and enclosed mass within a sphere of a given radius for total mass, stellar mass, and dark matter mass components). We also provide the complete models which can be used to compute the full luminous and mass distribution of each galaxy. Additionally, we visually assess the qualities of the models to help with model selections. We estimate the observed scatter in the measured quantities which decreases as expected with improvements in quality. For the best data quality, we find a remarkable consistency of measured quantities between different models, highlighting the robustness of the results.

Astronomy & Astrophysics↗

Visualization Quality Assessment

Understanding how inaccuracies in visualizations affect users’ perception and understanding of scientific data is hard. Inaccuracies in visualizations are quite common and could arise from a range of sources such as errors in the original dataset arising from compression artifacts, errors in the capturing device, noise during transmission of the data, effects due to the algorithm being used to convert data to visualization images, images generated from neural networks, and sources we have yet to discover. Many image quality assessment metrics have been developed to quantify image errors. However, these are usually focused on “natural images” rather than visualizations of scientific data. Common image quality assessment metrics (IQAs) include MSE, PSNR, perceptual metrics such SSIM, FSIM as well as perceptual metrics using deep learning approaches. However, a critical part of understanding how errors are perceived by humans, and subsequently developing more accurate quality assessment metrics, is through user evaluation studies. The goal of this software is to develop a visualization quality assessment (VQA) process that will enable the generation of VQAs that can be used to quantify errors in scientific data visualizations. The VQA development process will include software to support user evaluation experimental design, analysis of visualization differences against standard quality metrics, and the ability to develop additional VQA metrics specific to scientific visualization images.

Grosset, Andre↗

Quality Assessment of Frameless Fractionated Stereotactic Radiotherapy Using Cone Beam Computed Tomography

Purpose: A quality assessment of intracranial stereotactic radiotherapy was performed using cone beam computed tomography (CBCT). Setup errors were analyzed for two groups of patients: (1) those who were positioned using a frameless SonArray (FSA) system and immobilized with a bite plate and thermoplastic (TP) mask (the bFSA group); and (2) those who were positioned by room laser and immobilized using a TP mask (the mLAS group). Methods and Materials: A quality assurance phantom was used to study the system differences between FSA and CBCT. The quality assessment was performed using an Elekta Synergy imager (XVI) (Elekta Oncology Systems, Norcross, GA) and an On-Board Imager (OBI) (Varian Medical Systems, Palo Alto, CA) for 25 patients. For the first three fractions, and weekly thereafter, the FSA system was used for patient positioning, after which CBCT was performed to obtain setup errors. Results: (1) Phantom tests: The mean differences in the isocenter displacements for the two systems was 1.2 {+-} 0.7 mm. No significant variances were seen between the XVI and OBI units (p{approx}0.208). (2)Patient tests: The mean of the displacements between FSA and CBCT were independent of the CBCT system used; mean setup errors for the bFSA group were smaller (1.2 mm) than those of the mLAS group (3.2 mm) (p < 0.005). For the mLAS patients, the 90th percentile and the maximum rotational displacements were 3{sup o} and 5{sup o}, respectively. A 4-mm drift in setup accuracy occurred over the treatment course for 1 bFSA patient. Conclusions: System differences of less than 1 mm between CBCT and FSA were seen. Error regression was observed for the bFSA patients, using CBCT (up to 4 mm) during the treatment course. For the mLAS group, daily CBCT imaging was needed to obtain acceptable setup accuracies.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

SU-F-J-178: A Computer Simulation Model Observer for Task-Based Image Quality Assessment in Radiation Therapy

Purpose: Traditionally, image quality in radiation therapy is assessed subjectively or by utilizing physically-based metrics. Some model observers exist for task-based medical image quality assessment, but almost exclusively for diagnostic imaging tasks. As opposed to disease diagnosis, the task for image observers in radiation therapy is to utilize the available images to design and deliver a radiation dose which maximizes patient disease control while minimizing normal tissue damage. The purpose of this study was to design and implement a new computer simulation model observer to enable task-based image quality assessment in radiation therapy. Methods: A modular computer simulation framework was developed to resemble the radiotherapy observer by simulating an end-to-end radiation therapy treatment. Given images and the ground-truth organ boundaries from a numerical phantom as inputs, the framework simulates an external beam radiation therapy treatment and quantifies patient treatment outcomes using the previously defined therapeutic operating characteristic (TOC) curve. As a preliminary demonstration, TOC curves were calculated for various CT acquisition and reconstruction parameters, with the goal of assessing and optimizing simulation CT image quality for radiation therapy. Sources of randomness and bias within the system were analyzed. Results: The relationship between CT imaging dose and patient treatment outcome was objectively quantified in terms of a singular value, the area under the TOC (AUTOC) curve. The AUTOC decreases more rapidly for low-dose imaging protocols. AUTOC variation introduced by the dose optimization algorithm was approximately 0.02%, at the 95% confidence interval. Conclusion: A model observer has been developed and implemented to assess image quality based on radiation therapy treatment efficacy. It enables objective determination of appropriate imaging parameter values (e.g. imaging dose). Framework flexibility allows for incorporation of additional modules to include any aspect of the treatment process, and therefore has great potential for both assessment and optimization within radiation therapy.

60 APPLIED LIFE SCIENCES↗

IMPROVED AND QUALITY-ASSESSED EMISSION AND ABSORPTION LINE MEASUREMENTS IN SLOAN DIGITAL SKY SURVEY GALAXIES

We present a new database of absorption and emission-line measurements based on the entire spectral atlas from the Sloan Digital Sky Survey (SDSS) 7th data release of galaxies within a redshift of 0.2. Our work makes use of the publicly available penalized pixel-fitting (pPXF) and gas and absorption line fitting (gandalf) IDL codes, aiming to improve the existing measurements for stellar kinematics, the strength of various absorption-line features, and the flux and width of the emissions from different species of ionized gas. Our fit to the stellar continuum uses both standard stellar population models and empirical templates obtained by combining a large number of stellar spectra in order to fit a subsample of high-quality SDSS spectra for quiescent galaxies. Furthermore, our fit to the nebular spectrum includes an exhaustive list of both recombination and forbidden lines. Foreground Galactic extinction is implicitly treated in our models, whereas reddening in the SDSS galaxies is included in the form of a simple dust screen component affecting the entire spectrum that is accompanied by a second reddening component affecting only the ionized gas emission. In order to check for systematic departures from the rather standard set of assumptions that enters our models, we provide a quality assessment for our fit to the SDSS spectra in our sample, for both the stellar continuum and the nebular emissions and across different wavelength regions. This quality assessment also allows the identification of objects with either problematic data or peculiar features. We hope to foster the discovery potential of our database; therefore, our spectral fit is available to the community. For example, based on the quality assessment around the H{alpha} and [N II] {lambda}6584 lines, approximately 1% of the SDSS spectra classified as 'galaxies' by the SDSS pipeline do in fact require additional broad lines to be matched, even though they do not show a strong continuum from an active nucleus, as do the SDSS objects classified as 'quasars'. Finally, we provide new spectral templates for galaxies of different Hubble types, obtained by combining the results of our spectral fit for a subsample of 452 morphologically selected objects.

79 ASTRONOMY AND ASTROPHYSICS↗

SU-E-T-616: Plan Quality Assessment of Both Treatment Planning System Dose and Measurement-Based 3D Reconstructed Dose in the Patient

Purpose: Systematic radiotherapy plan quality assessment promotes quality improvement. Software tools can perform this analysis by applying site-specific structure dose metrics. The next step is to similarly evaluate the quality of the dose delivery. This study defines metrics for acceptable doses to targets and normal organs for a particular treatment site and scores each plan accordingly. The input can be the TPS or the measurement-based 3D patient dose. From this analysis, one can determine whether the delivered dose distribution to the patient receives a score which is comparable to the TPS plan score, otherwise replanning may be indicated. Methods: Eleven neuroblastoma patient plans were exported from Eclipse to the Quality Reports program. A scoring algorithm defined a score for each normal and target structure based on dose-volume parameters. Each plan was scored by this algorithm and the percentage of total possible points was obtained. Each plan also underwent IMRT QA measurements with a Mapcheck2 or ArcCheck. These measurements were input into the 3DVH program to compute the patient 3D dose distribution which was analyzed using the same scoring algorithm as the TPS plan. Results: The mean quality score for the TPS plans was 75.37% (std dev=14.15%) compared to 71.95% (std dev=13.45%) for the 3DVH dose distribution. For 3/11 plans, the 3DVH-based quality score was higher than the TPS score, by between 0.5 to 8.4 percentage points. Eight/11 plans scores decreased based on IMRT QA measurements by 1.2 to 18.6 points. Conclusion: Software was used to determine the degree to which the plan quality score differed between the TPS and measurement-based dose. Although the delivery score was generally in good agreement with the planned dose score, there were some that improved while there was one plan whose delivered dose quality was significantly less than planned. This methodology helps evaluate both planned and delivered dose quality. Sun Nuclear Corporation has provded a license for the software described.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

RFQAmodel: Random Forest Quality Assessment to identify a predicted protein structure in the correct fold

While template-free protein structure prediction protocols now produce good quality models for many targets, modelling failure remains common. For these methods to be useful it is important that users can both choose the best model from the hundreds to thousands of models that are commonly generated for a target, and determine whether this model is likely to be correct. We have developed Random Forest Quality Assessment (RFQAmodel), which assesses whether models produced by a protein structure prediction pipeline have the correct fold. RFQAmodel uses a combination of existing quality assessment scores with two predicted contact map alignment scores. These alignment scores are able to identify correct models for targets that are not otherwise captured. Our classifier was trained on a large set of protein domains that are structurally diverse and evenly balanced in terms of protein features known to have an effect on modelling success, and then tested on a second set of 244 protein domains with a similar spread of properties. When models for each target in this second set were ranked according to the RFQAmodel score, the highest-ranking model had a high-confidence RFQAmodel score for 67 modelling targets, of which 52 had the correct fold. At the other end of the scale RFQAmodel correctly predicted that for 59 targets the highest-ranked model was incorrect. In comparisons to other methods we found that RFQAmodel is better able to identify correct models for targets where only a few of the models are correct. We found that RFQAmodel achieved a similar performance on the model sets for CASP12 and CASP13 free-modelling targets. Finally, by iteratively generating models and running RFQAmodel until a model is produced that is predicted to be correct with high confidence, we demonstrate how such a protocol can be used to focus computational efforts on difficult modelling targets. RFQAmodel and the accompanying data can be downloaded from http://opig.stats.ox.ac.uk/resources.

59 BASIC BIOLOGICAL SCIENCES↗

Quality Assessment in Oncology

The movement to improve healthcare quality has led to a need for carefully designed quality indicators that accurately reflect the quality of care. Many different measures have been proposed and continue to be developed by governmental agencies and accrediting bodies. However, given the inherent differences in the delivery of care among medical specialties, the same indicators will not be valid across all of them. Specifically, oncology is a field in which it can be difficult to develop quality indicators, because the effectiveness of an oncologic intervention is often not immediately apparent, and the multidisciplinary nature of the field necessarily involves many different specialties. Existing and emerging comparative effectiveness data are helping to guide evidence-based practice, and the increasing availability of these data provides the opportunity to identify key structure and process measures that predict for quality outcomes. The increasing emphasis on quality and efficiency will continue to compel the medical profession to identify appropriate quality measures to facilitate quality improvement efforts and to guide accreditation, credentialing, and reimbursement. Given the wide-reaching implications of quality metrics, it is essential that they be developed and implemented with scientific rigor. The aims of the present report were to review the current state of quality assessment in oncology, identify existing indicators with the best evidence to support their implementation, and propose a framework for identifying and refining measures most indicative of true quality in oncologic care.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Practical procedures for sensor quality assessment

Sensors are increasingly deployed for process monitoring and control. These produce on-line measurements at a high frequency, in parallel with low-frequency laboratory measurements. Compared to laboratory practices, sensor data quality assessment and control practices are far less structured at most utilities. This leads to inaccurate sensor data with unknown uncertainty factors.This chapter shows how to establish standard operating procedures (SOPs) to support sensor data quality assessment and control and subsequent maintenance actions by producing relevant sensor metadata. Furthermore, SOPs are provided for the most commonly used wastewater quality sensors, inspired by utility and academic best practices. This chapter builds on definitions provided in Chapter 3 and provides additional definitions specifically related to sensors maintenance. Chapter 6 complements the methods in this chapter, which are based on reference measurements, with data-analytical techniques.

Alferes, Janelcy↗