Search NASA⌕ Search

SEARCH · Search NASA

Results for “quality assessment”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

3D-equivariant graph neural networks for protein model quality assessment

Quality assessment (QA) of predicted protein tertiary structure models plays an important role in ranking and using them. With the recent development of deep learning end-to-end protein structure prediction techniques for generating highly confident tertiary structures for most proteins, it is important to explore corresponding QA strategies to evaluate and select the structural models predicted by them since these models have better quality and different properties than the models predicted by traditional tertiary structure prediction methods. We develop EnQA, a novel graph-based 3D-equivariant neural network method that is equivariant to rotation and translation of 3D objects to estimate the accuracy of protein structural models by leveraging the structural features acquired from the state-of-the-art tertiary structure prediction method—AlphaFold2. We train and test the method on both traditional model datasets (e.g. the datasets of the Critical Assessment of Techniques for Protein Structure Prediction) and a new dataset of high-quality structural models predicted only by AlphaFold2 for the proteins whose experimental structures were released recently. Our approach achieves state-of-the-art performance on protein structural models predicted by both traditional protein structure prediction methods and the latest end-to-end deep learning method—AlphaFold2. It performs even better than the model QA scores provided by AlphaFold2 itself. The results illustrate that the 3D-equivariant graph neural network is a promising approach to the evaluation of protein structural models. Integrating AlphaFold2 features with other complementary sequence and structural features is important for improving protein model QA.

59 BASIC BIOLOGICAL SCIENCES↗

Rapid Quality Assessment in Additive Manufacturing: New Procedure Accelerates Build Quality Assessment. White paper

Accelerated growth of the additive manufacturing (AM) industry in recent years is accompanied by a rising need for methods to quickly assess quality at-scale. Current practices for quality inspection include nondestructive test methods and destructive testing of witness coupons, which are artifacts built alongside the actual part. However, these methods can be costly and time-consuming. Recognizing this need, the Additive Manufacturing Center of Excellence (AM CoE) initiated a project led by its partner, Auburn University, to develop rapid testing procedure using asbuilt samples tested in torsion to quantitatively assess build quality. The presented work developed a rapid testing procedure using as-built samples tested in torsion to quantify small variances for assessing build quality.

42 ENGINEERING↗

pyQuARC: Open Source Library for Earth Observation Metadata Quality Assessment

Metadata quality is essential to effective data discovery and has become increasingly vital as more Earth Science data sets become available. The Common Metadata Repository (CMR) hosts metadata describing NASA’s Earth Observation data products, which are archived across 12 Distributed Active Archive Centers (DAACs). The Analysis and Review of CMR (ARC) Team, located at Marshall Space Flight Center, conducts metadata quality assessments to ensure that these data products are discoverable, accessible, and usable. To achieve these goals, the ARC team has developed a metadata quality assessment framework to evaluate metadata completeness, correctness, and consistency. ARC uses a combination of manual and automated methods to assess these three components and identify areas of improvement; the team then collaborates with the DAACs to resolve any findings. To streamline this process, ARC is currently developing a host of scripts, known as pyQuARC, to automate metadata quality assessments as much as possible. pyQuARC is an open source library for Earth Observation Metadata Quality Assessment, and the tool utilizes ARC’s metadata quality assessment framework to make basic validation checks, pinpoint inconsistencies between dataset-level (i.e. collection) and file-level (i.e. granule) metadata, and identify opportunities for more descriptive and robust information. Since pyQuARC is also customizable, other users can make modifications as needed, and future metadata standards can also be implemented. Once pyQuARC is fully developed, it will support multiple schema types to serve the broader EOSDIS metadata community. This presentation will provide an overview of pyQuARC and its process of development while showcasing the tool’s valuable features and uses.

Jenny Wood↗

Commercial Smallsat Data Acquisition Program: Airbus U.S. Synthetic Aperture Radar Quality Assessment Summary

Quality assessment of the Airbus X-band Synthetic Aperture Radar (SAR) satellite products was conducted by the Commercial Smallsat Data Acquisition (CSDA) program’s radar subject matter experts, following the Joint NASA/ESA (European Space Agency) assessment draft guidelines. All three Airbus SAR spacecraft (TerraSAR-X, TanDEM-X, and PAZ) are based on the TerraSAR-X platform, and each have an active phased array antenna that is 4.8 x 0.7 m in the along-track and cross-track dimensions, respectively. TerraSAR-X and TanDEM-X are in a helical orbit, creating a bistatic imaging geometry, in addition to being capable of independent monostatic observations. The PAZ mission follows TerraSAR-X and TanDEM-X in the same 11-day orbit with a 5.5-day lag. TerraSAR-X and TanDEM-X are designed, developed, and operated through a Public-Private Partnership, while PAZ is a dual-use mission (civil and defense agencies), funded and owned by the Spanish Ministry of Defense and managed by Hisdesat (Hisdesat Servicios Estratégicos, S.A.), a Spanish private communications company. The assessment presented in this document is divided into two main parts: documentation review and the assessment of test datasets. The documentation review in sections 2.1 through 2.4 includes the assessment of the Airbus documentation provided to the CSDA evaluation team. The grading of these documents is given in columns 1-4 of the maturity matrix shown in section 1.1. Section 2.5 summarizes the evaluation performed by NASA using the data purchased through the CSDA program. The grading for this is given in the last column of the maturity matrix. Section 3 provides more detailed explanations on the methods and the results of the data analysis performed by NASA. Only the documents provided by Airbus for the evaluation were considered for the review. Additional documentation with more detailed description of the calibration and validation procedures may be available online but were not considered for this evaluation. The product information provided in the available documentation (RD-1, RD-2) and the product metadata together provided adequate information to work with the data. The product details in the metadata included the required information to work with the data in the common XML file format. Metrological traceability documentation was not provided to CSDA. All relevant characterization of the SAR system and data were provided, and the metadata include all relevant ancillary information. Documentation provided to CSDA included limited pre-flight and post-launch calibration information.

Batuhan Osmanoglu↗

Monitoring and data quality assessment of the ATLAS liquid argon calorimeter

The liquid argon calorimeter is a key component of the ATLAS detector installed at the CERN Large Hadron Collider. The primary purpose of this calorimeter is the measurement of electron and photon kinematic properties. It also provides a crucial input for measuring jets and missing transverse momentum. An advanced data monitoring procedure was designed to quickly identify issues that would affect detector performance and ensure that only the best quality data are used for physics analysis. This article presents the validation procedure developed during the 2011 and 2012 LHC data-taking periods, in which more than 98% of the proton-proton luminosity recorded by ATLAS at a centre-of-mass energy of 7-8 TeV had calorimeter data quality suitable for physics analysis.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Partially coherent wavefront propagation simulations: Mirror and monochromator crystal quality assessment

We have applied fully-and partially-coherent synchrotron radiation wavefront propagation simulations, implemented in the “Synchrotron Radiation Workshop” (SRW) computer code, to analyse the effects of imperfect mirrors and monochromator at the Coherent Hard X-ray beamline. This beamline is designed for X-ray Photon Correlation Spectroscopy, a technique that heavily relies on the partial coherence of the X-ray beam and benefits from a careful preservation of the X-ray wavefront. We present simulations and a comparison with the measured beam profile at the sample position, which show the impact of imperfect optics on the wavefront.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Environmental Justice Needs-Assessment For Health & Air Quality: Assessing the Needs and Capacity of Organizations Working Towards Health and Environmental Justice Related to Health and Air Quality

Health and air quality are concerns in environmental justice (EJ) work, as air contaminants and their adverse health effects inequitably impact communities that are often underrepresented in research and litigation. Previously, the NASA DEVELOP National Program partnered with organizations that focus on a variety of health and air quality concerns and is now committed to partnering directly with EJ organizations to address inequities in health and air quality through the application of Earth observations. To support the establishment of future partnerships, DEVELOP is working to identify potential partners for further EJ research in the U.S. regarding health and air quality. This project conducted a needs assessment with EJ organizations to quantify their EJ involvement, geospatial capacity, and interest in collaborating with DEVELOP. A broad literature review identified multiple factors influencing health and air quality for potential DEVELOP projects: agriculture, heat, wildfires, industry and transportation infrastructure, and miscellaneous factors (e.g., burning, urban heat island effect, smoke plumes, truck exhaust, and noise). The needs assessment showed that 22% of the organizations interviewed were primarily concerned with transportation infrastructure, 21% with industry, 19% with heat, 10% with miscellaneous factors (e.g., resource mining, COVID-19), 8% with wildfires, 7% with agriculture, and 6% with resource mining. This will help inform the DEVELOP Program's pursuit of EJ projects relating to health and air quality. The team also created a directory for communities impacted by air pollutants to connect with geospatial resources and NASA Applied Sciences personnel for future EJ health and air quality work.

Elizabeth Duran↗

Atmospheric Boundary Layer Wind Data During the Period January 1, 1998 Through January 31, 1999 at the Dallas-Fort Worth Airport: Quality Assessment - Volume 1

The quality of the Aircraft Vortex Spacing System (AVOSS) is critically dependent on representative wind profiles in the atmospheric boundary layer. These winds observed from a number of sensor systems around the Dallas-Fort Worth airport were combined into single vertical wind profiles by an algorithm developed and implemented by MIT Lincoln Laboratory. This process, called the AVOSS Winds Analysis System (AWAS), is used by AVOSS for wake corridor predictions. During times when AWAS solutions were available, the quality of the resultant wind profiles and variance was judged from a series of plots combining all sensor observations and AWAS profiles during the period 1200 to 0400 UTC daily. First, input data was evaluated for continuity and consistency from criteria established. Next, the degree of agreement among all wind sensor systems was noted and cases of disagreement identified. Finally, the resultant AWAS solution was compared to the quality-assessed input data. When profiles differed by a specified amount from valid sensor consensus winds, times and altitudes were flagged. Volume one documents the process and quality of input sensor data. Volume two documents the data processing/sorting process and provides the resultant flagged files.

Zak, J. Allen↗

Assessing the Hydrologic Performance of the EPA's Nonpoint Source Water Quality Assessment Decision Support Tool Using North American Land Data Assimilation System (Products)

The accuracy of streamflow predictions in the EPA's BASINS (Better Assessment Science Integrating Point and Nonpoint Sources) decision support tool is affected by the sparse meteorological data contained in BASINS. The North American Land Data Assimilation System (NLDAS) data with high spatial and temporal resolutions provide an alternative to the NOAA National Climatic Data Center (NCDC)'s station data. This study assessed the improvement of streamflow prediction of the Hydrological Simulation Program-FORTRAN (HSPF) model contained within BASINS using the NLDAS 118 degree hourly precipitation and evapotranspiration estimates in seven watersheds of the Chesapeake Bay region. Our results demonstrated consistent improvements of daily streamflow predictions in five of the seven watersheds when NLDAS precipitation and evapotranspiration data was incorporated into BASINS. The improvement of using the NLDAS data is significant when watershed's meteorological station is either far away or not in a similar climatic region. When the station is nearby, using the NLDAS data produces similar results. The correlation coefficients of the analyses using the NLDAS data were greater than 0.8, the Nash-Sutcliffe (NS) model fit efficiency greater than 0.6, and the error in the water balance was less than 5%. Our analyses also showed that the streamflow improvements were mainly contributed by the NLDAS's precipitation data and that the improvement from using NLDAS's evapotranspiration data was not significant; partially due to the constraints of current BASINS-HSPF settings. However, NLDAS's evapotranspiration data did improve the baseflow prediction. This study demonstrates the NLDAS data has the potential to improve stream flow predictions, thus aid the water quality assessment in the EPA nonpoint water quality assessment decision tool.

Lee, S.↗

A research review of quality assessment for software

Measures were recommended to assess the quality of software submitted to the AdaNet program. The quality factors that are important to software reuse are explored and methods of evaluating those factors are discussed. Quality factors important to software reuse are: correctness, reliability, verifiability, understandability, modifiability, and certifiability. Certifiability is included because the documentation of many factors about a software component such as its efficiency, portability, and development history, constitute a class for factors important to some users, not important at all to other, and impossible for AdaNet to distinguish between a priori. The quality factors may be assessed in different ways. There are a few quantitative measures which have been shown to indicate software quality. However, it is believed that there exists many factors that indicate quality and have not been empirically validated due to their subjective nature. These subjective factors are characterized by the way in which they support the software engineering principles of abstraction, information hiding, modularity, localization, confirmability, uniformity, and completeness.

Source record↗

Quality Assessment of GPU Power Profiling Mechanisms

Accurate component-level power measurements are nowadays essential for the design and optimization of high-performance computing (HPC) systems and applications. Particularly, as more and more heterogeneous HPC systems are developed, the characterizations of GPU power profiles have become extremely crucial because, although GPUs provide exceptional performance, they do consume substantial amounts of power. Currently, there are various GPU power profiling mechanisms available; however, there is no standard way to assess the quality of such profiling schemes. To address this issue, in this paper, we develop an assessment methodology to rate the quality and performance of the profiling mechanism itself. Specifically, we present the assessments of four different GPU power profiling techniques: (i) Nvidia's NVML via Allinea MAP, (ii) Nvidia's NVML via direct reads, and (iii) Penguin Computing's PowerInsight (PI) via two vendor-provided drivers, and (iv) PowerInsight via Allinea MAP. In addition, we discuss the effects of moving-average filters to explain the slow variations of some of the measured power profiles. Based on our assessment, the GPU power profiling mechanism using PI device outperforms the other schemes by reliably measuring the ground-truth power profile generated by a GPU stress-test benchmark.

Sen, Satyabrata↗

Analytical methods for online data quality assessment

This chapter provides a comprehensive overview of the main steps for algorithmic sensor signal quality assessment, which can enhance the decision-making process for water resource recovery facility (WRRF) operation and optimization. It introduces the concept of redundancy as the basis for data quality assessment. It also explains the typical data processing pipeline, which consists of preliminary analysis, data pre-processing, and specific algorithmic approaches. Each of these processes is presented and discussed in three separate sections. Importantly, this chapter introduces the main approaches for data quality assessment, provides guidelines for selecting the most suitable one and the key performance indicators to evaluate them and explains how to collect metadata through such an algorithmic approach.

Aguado, Daniel↗

A Refined Method to Translate Solar Data Quality Assessment Flags to Estimated Measurement Uncertainty

Integrating solar resource uncertainties due to radiometer measurement performance and operational data quality assessment can provide improved estimates of economic bankability, system design performance, and compliance of solar energy conversion systems. Estimating radiometer measurement uncertainty is an established procedure consistent with recognized best practices and international guidelines. SERI QC is a robust solar data quality assessment software tool that has been in continuous use for more than three decades. This report, the fourth of six for the Data Quality and Uncertainty Integration Project, presents a refined algorithm description for software to translate solar resource data quality assessment results into estimated uncertainty values in a Solar Resource Operational Uncertainty Integrator (SROUI) application. This algorithm requires three-component solar irradiance measurements - global horizontal irradiance, direct normal irradiance, and diffuse horizontal irradiance - collected at 1- to 60-minute intervals, as described in the previous deliverables. The development of this report as Deliverable 6.4 was an iterative process that included reviews and feedback from the project team on initial drafts designed to refine how the new software could best support determining solar resource data uncertainty. The results of this effort will contribute to the final software system development by National Renewable Energy Laboratory staff.

14 SOLAR ENERGY↗

Soil bioassays as tools for sludge compost quality assessment

Composting is a waste management technology that is becoming more widespread as a response to the increasing production of sewage sludge and the pressure for its reuse in soil. In this study, different bioassays (plant germination, earthworm survival, biomass and reproduction, and collembolan survival and reproduction) were assessed for their usefulness in the compost quality assessment. Compost samples, from two different composting plants, were taken along the composting process, which were characterized and submitted to bioassays (plant germination and collembolan and earthworm performance). Results from our study indicate that the noxious effects of some of the compost samples observed in bioassays are related to the low organic matter stability of composts and the enhanced release of decomposition endproducts, with the exception of earthworms, which are favored. Plant germination and collembolan reproduction inhibition was generally associated with uncomposted sludge, while earthworm total biomass and reproduction were enhanced by these materials. On the other hand, earthworm and collembolan survival were unaffected by the degree of composting of the wastes. However, this pattern was clear in one of the composting procedures assessed, but less in the other, where the release of decomposition endproducts was lower due to its higher stability, indicating the sensitivity and usefulness of bioassays for the quality assessment of composts.

09 BIOMASS FUELS↗

MaNGA DynPop – I. Quality-assessed stellar dynamical modelling from integral-field spectroscopy of 10K nearby galaxies: a catalogue of masses, mass-to-light ratios, density profiles, and dark matter

ABSTRACT This is the first paper in our series on the combined analysis of the Dynamics and stellar Population (DynPop) for the MaNGA survey in the final SDSS Data Release 17 (DR17). Here, we present a catalogue of dynamically determined quantities for over 10 000 nearby galaxies based on integral-field stellar kinematics from the MaNGA survey. The dynamical properties are extracted using the axisymmetric Jeans Anisotropic Modelling (JAM) method, which was previously shown to be the most accurate for this kind of study. We assess systematic uncertainties using eight dynamical models with different assumptions. We use two orientations of the velocity ellipsoid: either cylindrically aligned JAMcyl or spherically aligned JAMsph. We also make four assumptions for the models’ dark versus luminous matter distributions: (1) mass-follows-light, (2) free NFW dark halo, (3) cosmologically constrained NFW halo, (4) generalized NFW dark halo, i.e. with free inner slope. In this catalogue, we provide the quantities related to the mass distributions (e.g. the density slopes and enclosed mass within a sphere of a given radius for total mass, stellar mass, and dark matter mass components). We also provide the complete models which can be used to compute the full luminous and mass distribution of each galaxy. Additionally, we visually assess the qualities of the models to help with model selections. We estimate the observed scatter in the measured quantities which decreases as expected with improvements in quality. For the best data quality, we find a remarkable consistency of measured quantities between different models, highlighting the robustness of the results.

Astronomy & Astrophysics↗

Visualization Quality Assessment

Understanding how inaccuracies in visualizations affect users’ perception and understanding of scientific data is hard. Inaccuracies in visualizations are quite common and could arise from a range of sources such as errors in the original dataset arising from compression artifacts, errors in the capturing device, noise during transmission of the data, effects due to the algorithm being used to convert data to visualization images, images generated from neural networks, and sources we have yet to discover. Many image quality assessment metrics have been developed to quantify image errors. However, these are usually focused on “natural images” rather than visualizations of scientific data. Common image quality assessment metrics (IQAs) include MSE, PSNR, perceptual metrics such SSIM, FSIM as well as perceptual metrics using deep learning approaches. However, a critical part of understanding how errors are perceived by humans, and subsequently developing more accurate quality assessment metrics, is through user evaluation studies. The goal of this software is to develop a visualization quality assessment (VQA) process that will enable the generation of VQAs that can be used to quantify errors in scientific data visualizations. The VQA development process will include software to support user evaluation experimental design, analysis of visualization differences against standard quality metrics, and the ability to develop additional VQA metrics specific to scientific visualization images.

Grosset, Andre↗

Quality Assessment of Frameless Fractionated Stereotactic Radiotherapy Using Cone Beam Computed Tomography

Purpose: A quality assessment of intracranial stereotactic radiotherapy was performed using cone beam computed tomography (CBCT). Setup errors were analyzed for two groups of patients: (1) those who were positioned using a frameless SonArray (FSA) system and immobilized with a bite plate and thermoplastic (TP) mask (the bFSA group); and (2) those who were positioned by room laser and immobilized using a TP mask (the mLAS group). Methods and Materials: A quality assurance phantom was used to study the system differences between FSA and CBCT. The quality assessment was performed using an Elekta Synergy imager (XVI) (Elekta Oncology Systems, Norcross, GA) and an On-Board Imager (OBI) (Varian Medical Systems, Palo Alto, CA) for 25 patients. For the first three fractions, and weekly thereafter, the FSA system was used for patient positioning, after which CBCT was performed to obtain setup errors. Results: (1) Phantom tests: The mean differences in the isocenter displacements for the two systems was 1.2 {+-} 0.7 mm. No significant variances were seen between the XVI and OBI units (p{approx}0.208). (2)Patient tests: The mean of the displacements between FSA and CBCT were independent of the CBCT system used; mean setup errors for the bFSA group were smaller (1.2 mm) than those of the mLAS group (3.2 mm) (p < 0.005). For the mLAS patients, the 90th percentile and the maximum rotational displacements were 3{sup o} and 5{sup o}, respectively. A 4-mm drift in setup accuracy occurred over the treatment course for 1 bFSA patient. Conclusions: System differences of less than 1 mm between CBCT and FSA were seen. Error regression was observed for the bFSA patients, using CBCT (up to 4 mm) during the treatment course. For the mLAS group, daily CBCT imaging was needed to obtain acceptable setup accuracies.

62 RADIOLOGY AND NUCLEAR MEDICINE↗