Search NASA⌕ Search

SEARCH · Search NASA

Results for “data quality assessment”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

Improved Quality Control Processing of Peptide-centric LC-MS Proteomics Data

In the analysis of differential peptide peak intensities (i.e., abundance measures), LC-MS analyses with poor quality peptide abundance data can bias downstream statistical analyses and hence the biological interpretation for an otherwise high quality data set. Although considerable effort has been placed on assuring the quality of the peptide identification with respect to spectral processing, to date quality assessment of the subsequent peptide abundance data matrix has been limited to a subjective visual inspection of run-by-run correlation or individual peptide components. Identifying statistical outliers is a critical step in the processing of proteomics data as many of the downstream statistical analyses (e.g., ANOVA) rely upon accurate estimates of sample variance, and their results are influenced by extreme values. Results: We describe a multivariate statistical strategy for the identification of LC-MS runs with extreme peptide abundance distributions. Comparison with current method (run-by-run correlation) demonstrates the significantly better rate of identification of outlier runs by the multivariate strategy. Simulation studies also suggest this strategy significantly outperforms correlation alone in the identification of statistically extreme LC-MS runs.

59 BASIC BIOLOGICAL SCIENCES↗

A Unified Approach for Reporting ARM Measurement Uncertainties Technical Report

The U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Climate Research Facility is observationally based, and quantifying the uncertainty of its measurements is critically important. With over 300 widely differing instruments providing over 2,500 datastreams, concise expression of measurement uncertainty is quite challenging. The ARM Facility currently provides data and supporting metadata (information about the data or data quality) to its users through a number of sources. Because the continued success of the ARM Facility depends on the known quality of its measurements, the Facility relies on instrument mentors and the ARM Data Quality Office (DQO) to ensure, assess, and report measurement quality. Therefore, an easily accessible, well-articulated estimate of ARM measurement uncertainty is needed. Note that some of the instrument observations require mathematical algorithms (retrievals) to convert a measured engineering variable into a useful geophysical measurement. While those types of retrieval measurements are identified, this study does not address particular methods for retrieval uncertainty. As well, the ARM Facility also provides engineered data products, or value-added products (VAPs), based on multiple instrument measurements. This study does not include uncertainty estimates for those data products. We propose here that a total measurement uncertainty should be calculated as a function of the instrument uncertainty (calibration factors), the field uncertainty (environmental factors), and the retrieval uncertainty (algorithm factors). The study will not expand on methods for computing these uncertainties. Instead, it will focus on the practical identification, characterization, and inventory of the measurement uncertainties already available in the ARM community through the ARM instrument mentors and their ARM instrument handbooks. As a result, this study will address the first steps towards reporting ARM measurement uncertainty: 1) identifying how the uncertainty of individual ARM measurements is currently expressed, 2) identifying a consistent approach to measurement uncertainty, and then 3) reclassifying ARM instrument measurement uncertainties in a common framework.

47 OTHER INSTRUMENTATION↗

Open Rotor Tone Shielding Methods for System Noise Assessments Using Multiple Databases

Advanced aircraft designs such as the hybrid wing body, in conjunction with open rotor engines, may allow for significant improvements in the environmental impact of aviation. System noise assessments allow for the prediction of the aircraft noise of such designs while they are still in the conceptual phase. Due to significant requirements of computational methods, these predictions still rely on experimental data to account for the interaction of the open rotor tones with the hybrid wing body airframe. Recently, multiple aircraft system noise assessments have been conducted for hybrid wing body designs with open rotor engines. These assessments utilized measured benchmark data from a Propulsion Airframe Aeroacoustic interaction effects test. The measured data demonstrated airframe shielding of open rotor tonal and broadband noise with legacy F7/A7 open rotor blades. Two methods are proposed for improving the use of these data on general open rotor designs in a system noise assessment. The first, direct difference, is a simple octave band subtraction which does not account for tone distribution within the rotor acoustic signal. The second, tone matching, is a higher-fidelity process incorporating additional physical aspects of the problem, where isolated rotor tones are matched by their directivity to determine tone-by-tone shielding. A case study is conducted with the two methods to assess how well each reproduces the measured data and identify the merits of each. Both methods perform similarly for system level results and successfully approach the experimental data for the case study. The tone matching method provides additional tools for assessing the quality of the match to the data set. Additionally, a potential path to improve the tone matching method is provided.

Bahr, Christopher J.↗

Comparison of Integral Parameter Estimated Uncertainties in Fast Neutron Systems with COMMARA-2.1 and COMMARA-2.0 Covariance Data

This paper shows the main differences between the COMMARA-2.0 and COMMARA-2.1 evaluated covariance data in the uncertainty estimation of integral parameters of interest for a large number of typical innovative fast neutron systems. The study provides some guidelines to the nuclear data evaluators on which materials and which reaction types need to be evaluated most accurately to achieve reliable results in computational analyses of similar systems. The assessment of the quality and reliability of the covariance data is outside the scope of the study.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Validation of the Cloud_CCI (Cloud Climate Change Initiative) cloud products in the Arctic

The role of clouds in the Arctic radiation budget is not well understood. Ground-based and airborne measurements provide valuable data to test and improve our understanding. However, the ground-based measurements are intrinsically sparse, and the airborne observations are snapshots in time and space. Passive remote sensing measurements from satellite sensors offer high spatial coverage and an evolving time series, having lengths potentially of decades. However, detecting clouds by passive satellite remote sensing sensors is challenging over the Arctic because of the brightness of snow and ice in the ultraviolet and visible spectral regions and because of the small brightness temperature contrast to the surface. Consequently, the quality of the resulting cloud data products needs to be assessed quantitatively. In this study, we validate the cloud data products retrieved from the Advanced Very High Resolution Radiometer (AVHRR) post meridiem (PM) data from the polar-orbiting NOAA-19 satellite and compare them with those derived from the ground-based instruments during the sunlit months. The AVHRR cloud data products by the European Space Agency (ESA) Cloud Climate Change Initiative (Cloud_CCI) project uses the observations in the visible and IR bands to determine cloud properties. The ground-based measurements from four high-latitude sites have been selected for this investigation: Hyytiälä (61.84°N, 24.29°E), North Slope of Alaska (NSA; 71.32°N, 156.61°W), Ny-Ålesund (Ny-Å; 78.92°N, 11.93°E), and Summit (72.59°N, 38.42°W). The liquid water path (LWP) ground-based data are retrieved from microwave radiometers, while the cloud top height (CTH) has been determined from the integrated lidar–radar measurements. The quality of the satellite products, cloud mask and cloud optical depth (COD), has been assessed using data from NSA, whereas LWP and CTH have been investigated over Hyytiälä, NSA, Ny-Å, and Summit.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of Wind Power Forecasts from the Vermont Weather Analytics Center and Identification of Improvements

The goal of this analysis was to assess the wind power forecast accuracy of the Vermont Weather Analytics Center (VTWAC) forecast system and to identify potential improvements to the forecasts. Based on the analysis at Georgia Mountain, the following recommendations for improving forecast performance were made: 1. Resolve the significant negative forecast bias in February-March 2017 (50% underprediction on average) 2. Improve the ability of the forecast model to capture the strong diurnal cycle of wind power 3. Add ability for forecast model to assess internal wake loss, particularly at sites where strong diurnal shifts in wind direction are present. Data availability and quality limited the robustness of this forecast assessment. A more thorough analysis would be possible given a longer period of record for the data (at least one full year), detailed supervisory control and data acquisition data for each wind plant, and more detailed information on the forecast system input data and methodologies.

17 WIND ENERGY↗

Aero-thermal Calibration of the NASA Glenn Icing Research Tunnel (2000 Tests)

Aerothermal calibration measurements and flow quality surveys were made in the test section of the Icing Research Tunnel at the NASA Glenn Research Center. These surveys were made following major facility modifications including widening of the heat exchanger tunnel section, replacement of the heat exchanger, installation of new turning vanes, and installation of new fan exit guide vanes. Standard practice at NASA Glenn requires that test section calibration and flow quality surveys be performed following such major facility modifications. A single horizontally oriented rake was used to survey the flow field at several vertical positions within a single cross-sectional plane of the test section. These surveys provided a detailed mapping of the total and static pressure, total temperature, Mach number, velocity, flow angle and turbulence intensity. Data were acquired over the entire velocity and total temperature range of the facility. No icing conditions were tested; however, the effects of air sprayed through the water injecting spray bars were assessed. All data indicate good flow quality. Mach number standard deviations were less than 0.0017, flow angle standard deviations were between 0.3 deg and 0.8 deg, total temperature standard deviations were between 0.5 and 1.8 F for subfreezing conditions, axial turbulence intensities varied between 0.3 and 1.0 percent, and transverse turbulence intensities varied between 0.3 and 1.5 percent. Measurement uncertainties were also quantified.

Gonsalez, Jose C.↗

Using CALIPSO's New Ocean Derived Column Optical Depths

CALIPSO’s Version 4.51 Level 2 data release introduces an all-new group of science data sets containing estimates of total column two-way transmittances and effective optical depths derived from CALIOP ocean surface backscatter measurements and MERRA-2 reanalysis wind speed data. These estimates use data from the standard CALIOP lidar signal but in a passive sensor-like way, thus creating a unique link to passive instrument measurements. These new retrievals are provided for the entire mission, day and night, at 532nm, and are reported at single shot, 1km, and 5km resolutions for all profiles in which a valid lidar ocean surface return is detected. The addition of a total column optical depth constraint on subsequent retrievals of cloud and aerosol optical properties opens the door for many exciting new ways to leverage the already rich and versatile CALIPSO dataset. Following a brief review of the retrieval technique, this talk will focus on quality assurance assessments, estimated uncertainties, and data usage scenarios. We will conclude with examples highlighting some of the exciting work already being done using this new addition to CALIPSO’s already rich data record.

R Ryan↗

Critical statistical assessment of data in metal additive manufacturing

Obtaining high quality data reflecting the relationships between the additive manufacturing (AM) process parameters, material microstructure and mechanical properties is crucial for the use of machine learning in AM. A database of over 4,000 data entries of metal AM was created thanks to a large number of literature studies on key process parameters and indicators of build quality. Meta-analysis reveals critical biases in the literature. Firstly, majority of studies report only high quality builds, these imbalances in reporting result in weak correlation between process parameters, properties and consolidation, limiting the ability of machine learning models to generalize beyond optimized conditions. Nevertheless, the trained models accurately predict yield strength ($R^2 = 0.85$), suggesting that certain process–property relationships are effectively captured within these models. Secondly, quantitative microstructural data are largely absent, limiting the learning of the microstructure-mechanical properties relationships. Finally, current process window identification is based largely on the consolidation, despite significant uncertainty in its measurement. It is important to identify the process map on the basis of not only the consolidation, but also mechanical behaviour under loading. Such a identification shows that 316 L and Inconel have much larger process map (i.e. highly printable) in comparison to the AlSi10Mg and Ti6Al4V.

Additive manufacturing↗

Assessment of satellite communications quality study. Addendum 1: Impact of propagation delay on data transmission

The single factor that irrevocably distinguishes geostationary satellite telephony transmission from terrestrial transmission is the greater propagation delay over satellite links. This difference has always provoked vigorous debate over the impact of delay on the subscribers using services incorporating satellite links. The issue is addressed from a variety of directions including human factors studies, laboratory subjective tests that evaluate delay with and without echo, and field tests that obtain data on the opinion of subscribers regarding the quality of service of operational circuits in both national U.S. domestic and international trans-Atlantic network. The tests involved the use of both echo suppressors and echo cancellers.

Campanella, S. J.↗

Deriving Daytime Variables From the AmeriFlux Standard Eddy Covariance Data Set

A gap-filled, quality assessed eddy covariance dataset has recently become available for the AmeriFluxnetwork. This dataset uses standard processing and produces commonly used science variables. This shared dataset enables robust comparisons across different analyses. Of course, there are many remaining questions. One of those is how to define 'during the day' which is an important concept for many analyses. Some studies have used local time — for example 9am to 5pm; others have used thresholds on photosynthetic active radiation (PAR). A related question is how to derive quantities such as the Bowen ratio. Most studies compute the ratio of the averages of the latent heat (LE) and sensible heat (H). In this study, we use different methods of defining 'during the day' for GPP, LE, and H. We evaluate the differences between methods in two ways. First, we look at a number of statistics of GPP. Second, we look at differences in the derived Bowen ratio. Our goal is not science per se, but rather informatics in support of the science.

54 ENVIRONMENTAL SCIENCES↗

The Atlanta Urban Heat Island Mitigation and Air Quality Modeling Project: How High-Resoution Remote Sensing Data Can Improve Air Quality Models

The Atlanta Urban Heat Island and Air Quality Project had its genesis in Project ATLANTA (ATlanta Land use Analysis: Temperature and Air quality) that began in 1996. Project ATLANTA examined how high-spatial resolution thermal remote sensing data could be used to derive better measurements of the Urban Heat Island effect over Atlanta. We have explored how these thermal remote sensing, as well as other imaged datasets, can be used to better characterize the urban landscape for improved air quality modeling over the Atlanta area. For the air quality modeling project, the National Land Cover Dataset and the local scale Landpro99 dataset at 30m spatial resolutions have been used to derive land use/land cover characteristics for input into the MM5 mesoscale meteorological model that is one of the foundations for the Community Multiscale Air Quality (CMAQ) model to assess how these data can improve output from CMAQ. Additionally, land use changes to 2030 have been predicted using a Spatial Growth Model (SGM). SGM simulates growth around a region using population, employment and travel demand forecasts. Air quality modeling simulations were conducted using both current and future land cover. Meteorological modeling simulations indicate a 0.5 C increase in daily maximum air temperatures by 2030. Air quality modeling simulations show substantial differences in relative contributions of individual atmospheric pollutant constituents as a result of land cover change. Enhanced boundary layer mixing over the city tends to offset the increase in ozone concentration expected due to higher surface temperatures as a result of urbanization.

Quattrochi, Dale A.↗

A 3D MR-acquisition scheme for nonrigid bulk motion correction in simultaneous PET-MR

Purpose: Positron emission tomography (PET) is a highly sensitive medical imaging technique commonly used to detect and assess tumor lesions. Magnetic resonance imaging (MRI) provides high resolution anatomical images with different contrasts and a range of additional information important for cancer diagnosis. Recently, simultaneous PET-MR systems have been released with the promise to provide complementary information from both modalities in a single examination. Due to long scan times, subject nonrigid bulk motion, i.e., changes of the patient's position on the scanner table leading to nonrigid changes of the patient's anatomy, during data acquisition can negatively impair image quality and tracer uptake quantification. A 3D MR-acquisition scheme is proposed to detect and correct for nonrigid bulk motion in simultaneously acquired PET-MR data. Methods: A respiratory navigated three dimensional (3D) MR-acquisition with Radial Phase Encoding (RPE) is used to obtain T1- and T2-weighted data with an isotropic resolution of 1.5 mm. Healthy volunteers are asked to move the abdomen two to three times during data acquisition resulting in overall 19 movements at arbitrary time points. The acquisition scheme is used to retrospectively reconstruct dynamic 3D MR images with different temporal resolutions. Nonrigid bulk motion is detected and corrected in this image data. A simultaneous PET acquisition is simulated and the effect of motion correction is assessed on image quality and standardized uptake values (SUV) for lesions with different diameters. Results: Six respiratory gated 3D data sets with T1- and T2-weighted contrast have been obtained in healthy volunteers. All bulk motion shifts have successfully been detected and motion fields describing the transformation between the different motion states could be obtained with an accuracy of 1.71 ± 0.29 mm. The PET simulation showed errors of up to 67% in measured SUV due to bulk motion which could be reduced to less than 10% with the proposed motion compensation approach. Conclusions: A MR acquisition scheme which yields both high resolution 3D anatomical data and highly accurate nonrigid motion information without an increase in scan time is presented. The proposed method leads to a strong improvement in both MR and PET image quality and ensures an accurate assessment of tracer uptake.

60 APPLIED LIFE SCIENCES↗

In-situ qualification and physics-based process design for aerosol jet printing via spatially correlated light scattering measurements

Aerosol jet printing is a contactless, digital, and additive technique broadly used for manufacturing flexible, hybrid, and conformal electronics. However, batch-to-batch variability has hindered widespread industry adoption and scaling to production volumes. Recently, light scattering measurements have emerged as a tool to measure aerosol volume fraction – a key parameter determining deposition rate – and have proven an effective feedback source for closed loop control on timescales ranging from minutes to hours. Here, the efficacy of light scattering measurements as a tool for in-situ qualification over shorter time durations is explored. To be the linear relationship between light scattering measurements and deposition rate was validated at 500 ms time intervals, allowing deposition rate to be mapped to positional coordinates and providing a new data stream to drive quality control assessments. Because light scattering measurements are indicative of a physical process parameter, they were substituted into equations for resistance and sheet resistance, resulting in predictions nominally within 10% of measured values for sets of printed devices. Finally, to highlight a more active utility, light scattering measurements were employed in a print repair framework, successfully repairing prints with randomly induced defects to within 5% of the measured resistance of a control set.

36 MATERIALS SCIENCE↗

Does Quality of Radiation Therapy Predict Outcomes of Multicenter Cooperative Group Trials? A Literature Review

Central review of radiation therapy (RT) delivery within multicenter clinical trials was initiated in the early 1970s in the United States. Early quality assurance publications often focused on metrics related to process, logistics, and timing. Our objective was to review the available evidence supporting correlation of RT quality with clinical outcomes within cooperative group trials. A MEDLINE search was performed to identify multicenter studies that described central subjective assessment of RT protocol compliance (quality). Data abstracted included method of central review, definition of deviations, and clinical outcomes. Seventeen multicenter studies (1980-2012) were identified, plus one Patterns of Care Study. Disease sites were hematologic, head and neck, lung, breast, and pancreas. Between 0 and 97% of treatment plans received an overall grade of acceptable. In 7 trials, failure rates were significantly higher after inadequate versus adequate RT. Five of 9 and 2 of 5 trials reported significantly worse overall and progression-free survival after poor-quality RT, respectively. One reported a significant correlation, and 2 reported nonsignificant trends toward increased toxicity with noncompliant RT. Although more data are required, protocol-compliant RT may decrease failure rates and increase overall survival and likely contributes to the ability of collected data to answer the central trial question.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Solar Radiometer Instrumentation Evaluation: Cooperative Research and Development (Final Report, CRADA Number CRD-16-00619)

The purpose and intent of this agreement is to evaluate newly manufactured thermopile pyranometers and spectroradiometers. The purpose also extends to provide a framework for Participant to conduct research to improve and develop new radiometric devices and application in the future. The overall objective is to provide more accurate, site-specific, long-term, continuous measurements of the solar resources needed by industry to increase the deployment and improve the operations of photovoltaic and concentrating solar power plants. This CRADA addresses the needs for proven solar irradiance measurement to validate resource assessment models and generate high quality data used for site selection, energy system design, deployment, maintenance, and operation. This work will be conducted at NREL and Participant facilities. This project will place instrumentation at the NREL Solar Radiation Research Laboratory (SRRL) in cooperation with EKO Instruments, USA. Participant instruments will be deployed for the purpose of evaluation under controlled conditions. The scope of the project will be a 3 years-long segmented comparison of the instruments vs. other NREL baseline instruments with a well-characterized history. These evaluations will include planned improvements to instruments as well as extensions of future instrument applications. The final evaluation will be a written report similar to the work done by Wilcox and Myers (see http://www.nrel.gov/docs/fy09osti/44627.pdf).

14 SOLAR ENERGY↗

An Overview of the Molten Salt Thermal Properties Database–Thermophysical, Version 4.0 (MSTDB-TP V.4.0)

A central repository of thermophysical and thermochemical properties of molten salt compositions of relevance to molten salt reactors (MSRs) is vital in supporting the broad community of MSR developers, who are at various stages of developing and deploying their reactor designs. In general, these MSR designs differ significantly from developer to developer (e.g., with respect to the hardness of the neutron spectra, level of fissile loading, target multicomponent temperatures and power levels, and moderating capabilities). Therefore, the fuel and coolant salts being considered vary greatly: they may be chlorides or fluorides, they utilize different actinides at different ratios, and the cations in the melt are selected based on perceived advantages and disadvantages. Considering the general need for thermal properties, and the vastness of the array of potential candidate salt mixtures, the Molten Salt Thermal Properties Database (MSTDB) was initiated in 2018 with the goal of providing thermophysical and thermochemical characterization of key molten salt compounds and mixtures across their temperature and compositional domains. The MSTDB is thus divided into the thermophysical arm (MSTDB-TP) and the thermochemical arm (MSTDB-TC). The MSTDB is an effort funded by the Department of Energy, Office of Nuclear Energy (DOE-NE) Nuclear Energy Advanced Modeling and Simulation (NEAMS) program, and the MSR Campaign. This report provides an overview of the MSTDB-TP v4.0 in terms of the data contained within, the state of the tools used to access the data, the availability of predictive models that leverage the raw data in the database, the preliminary status of developmental efforts that are currently underway, and an account of future goals for MSTDB-TP. The primary goal for the update from MSTDB-TP v.3.1 to v4.0 was the incorporation of surface tension data into the database; this property is important for thermal hydraulics modeling and species transport in other tools that have been developed under the NEAMS program. A breakdown of the surface tension data that have been added into MSTDB-TP v4.0 is provided herein, and the manner in which the quality of the data has been assessed is also documented. For MSTDB-TP v4.0, newly published thermophysical property data—primarily from collaborative experimental efforts under the MSR Campaign—have been incorporated into the database, and the resulting expansion is documented here. Because of the size to which MSTDB-TP has grown, the raw data format has now been recast into JavaScript Object Notation (JSON) format for easier connection with the MSTDB-TP application programming interface (API). Saline; the pre-existing comma-separated value (CSV) format has been deprecated but is still maintained, accessible, and up to date. As a final effort in packaging the MSTDB-TP v4.0 update, the graphical user interface (GUI) for MSTDB has been updated to allow full accessibility to the density and viscosity predictive models, which are based on Redlich-Kister expansions of MSTDB-TP raw data. Some other major aspects of this report, in terms of preliminary and future work, include: (1) documentation of the formalism and preliminary testing of a kinetic theory model that may act as a predictive model for thermal conductivity; (2) documentation of the candidate predictive models that may be considered in the future for surface tension, making use of the surface tension data now in MSTDB-TP v4.0; (3) a preliminary account of a data collection process that will enable the filling of additional gaps within MSTDB-TP, namely with data which have been collected computationally (e.g., through ab initio molecular dynamics).

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Implications of Wide-Area Geographic Diversity for Short- Term Variability of Solar Power

Worldwide interest in the deployment of photovoltaic generation (PV) is rapidly increasing. Operating experience with large PV plants, however, demonstrates that large, rapid changes in the output of PV plants are possible. Early studies of PV grid impacts suggested that short-term variability could be a potential limiting factor in deploying PV. Many of these early studies, however, lacked high-quality data from multiple sites to assess the costs and impacts of increasing PV penetration. As is well known for wind, accounting for the potential for geographic diversity can significantly reduce the magnitude of extreme changes in aggregated PV output, the resources required to accommodate that variability, and the potential costs of managing variability. We use measured 1-min solar insolation for 23 time-synchronized sites in the Southern Great Plains network of the Atmospheric Radiation Measurement program and wind speed data from 10 sites in the same network to characterize the variability of PV with different degrees of geographic diversity and to compare the variability of PV to the variability of similarly sited wind. The relative aggregate variability of PV plants sited in a dense 10 x 10 array with 20 km spacing is six times less than the variability of a single site for variability on time scales less than 15-min. We find in our analysis of wind and PV plants similarly sited in a 5 x 5 grid with 50 km spacing that the variability of PV is only slightly more than the variability of wind on time scales of 5-15 min. Over shorter and longer time scales the level of variability is nearly identical. Finally, we use a simple approximation method to estimate the cost of carrying additional reserves to manage sub-hourly variability. We conclude that the costs of managing the short-term variability of PV are dramatically reduced by geographic diversity and are not substantially different from the costs for managing the short-term variability of similarly sited wind in this region.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗