Search NASA⌕ Search

SEARCH · Search NASA

Results for “data quality metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Astronaut Sleep Duration Varies by Timing of Scheduled Sleep

INTRODUCTION: Studies find that humans average approximately six hours of sleep per night in space, which is less than they sleep on Earth. Such short sleep duration has been associated with reduced alertness and performance in space. It is unclear whether this sleep loss is related to modifiable factors, such as irregular scheduling, poor sleep environment, and excessive workload or due to features of spaceflight that alter physiology (e.g., microgravity). Recent missions have afforded crew better, more stable sleep and work schedules, and an improved sleep environment. Despite these improvements, schedules do still vary enough to cause decrements in sleep duration. METHODS: Crewmembers (n = 19) who volunteered for the NASA Standard Measures protocol between January 2019 and March 2022 were provided with actiwatches (Phillips, Respironics, Bend OR) that they wore for two bouts of data collection lasting two weeks each before flight (at approximately L-270 and L-180), either continuously (n = 9) or for two weeks every two months while in space (n = 10), and for seven days postflight, immediately upon return to Earth (R+0). A regularly scheduled (or “nominal”) sleep episode would take place between the hours of 9:30pm and 6:00am. We looked at sleep outcomes (sleep duration, wake after sleep onset [WASO], sleep efficiency) depending on the distance from nominal sleep offset to see whether scheduled sleep period affected sleep durations and other metrics of sleep quality. RESULTS: Crewmembers provided data from 402 nights preflight, 2,137 nights inflight, and 275 nights postflight. They averaged 7.33 hours of sleep per night (± 1.16, SD) in space. Though this was significantly less sleep than they achieved preflight (7.87 ± 1.10) or postflight (7.75 ± 1.43, p < .01), this duration of sleep meets the recommended amount for optimal human health and well-being. For every hour after the nominal sleep period a crewmember woke up, their total sleep increased by 25 minutes (up to 5 hours). CONCLUSIONS: We conclude that humans are capable of achieving sufficient sleep in space, especially when their schedules afford adequate sleep (namely, schedules that phase delay rather than advance). Future studies are needed to determine whether microgravity impacts sleep architecture and sleep quality. Going forward, it is imperative that crewmembers are provided with stable schedules, with moderate workload, and environments that are conducive to sleep.

fatigue↗

Science Resource Office Peer Review and Metrics Factsheet [Slides]

The Science Resource Office-Peer Review and Metrics (SRO-PRM) Team hosts and maintains LANL's central repository for Science, Technology, and Engineering (STE) data. The Team tracks, integrates, analyzes, and reports qualitative and quantitative data measuring performance, quality, impact, and productivity. These metrics are indicators of the effectiveness and impact of STE efforts.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Testing of the Apollo 15 Metric Camera System.

Description of tests conducted (1) to assess the quality of Apollo 15 Metric Camera System data and (2) to develop production procedures for total block reduction. Three strips of metric photography over the Hadley Rille area were selected for the tests. These photographs were utilized in a series of evaluation tests culminating in an orbitally constrained block triangulation solution. Results show that film deformations up to 25 and 5 microns are present in the mapping and stellar materials, respectively. Stellar reductions can provide mapping camera orientations with an accuracy that is consistent with the accuracies of other parameters in the triangulation solutions. Pointing accuracies of 4 to 10 microns can be expected for the mapping camera materials, depending on variations in resolution caused by changing sun angle conditions.

Helmering, R. J.↗

A User-Facing Metric to Quantify the Quality of Mobility (CRADA Final Report)

The leading urban mobility data analytics firm StreetLight Data, Inc. partnered with the National Renewable Energy Laboratory to explore a commercial version of the Mobility Energy Productivity (MEP) metric. The commercialization effort was aimed to expand the adoption of the metric to key stakeholders in the urban planning space. Research comprised industry analysis, stakeholder feedback and conducting transportation practitioner focus groups. StreetLight concluded that commercialization of MEP is not feasible in the current market because users need a dynamic MEP tool that enables scenario planning. It should be able to calculate a MEP score dynamically (near instantaneous) when different inputs are changed.

33 ADVANCED PROPULSION SYSTEMS↗

Analysis of Sting Balance Calibration Data Using Optimized Regression Models

Calibration data of a wind tunnel sting balance was processed using a candidate math model search algorithm that recommends an optimized regression model for the data analysis. During the calibration the normal force and the moment at the balance moment center were selected as independent calibration variables. The sting balance itself had two moment gages. Therefore, after analyzing the connection between calibration loads and gage outputs, it was decided to choose the difference and the sum of the gage outputs as the two responses that best describe the behavior of the balance. The math model search algorithm was applied to these two responses. An optimized regression model was obtained for each response. Classical strain gage balance load transformations and the equations of the deflection of a cantilever beam under load are used to show that the search algorithm s two optimized regression models are supported by a theoretical analysis of the relationship between the applied calibration loads and the measured gage outputs. The analysis of the sting balance calibration data set is a rare example of a situation when terms of a regression model of a balance can directly be derived from first principles of physics. In addition, it is interesting to note that the search algorithm recommended the correct regression model term combinations using only a set of statistical quality metrics that were applied to the experimental data during the algorithm s term selection process.

Ulbrich, N.↗

An Overall Assessment of JPSS-3 VIIRS Radiometric Performance Based on Pre-Launch Testing

Satellite imagery and data are playing an increasingly important role in scientific studies of the Earth and its climate. The scientific community has been demanding ever-increasing capabilities and accuracy from the data provided by these satellites. One key instrument on board a series of satellite platforms is the Visible Infrared Imaging Radiometer Suite (VIIRS), which provides high-quality data of the Earth from low Earth orbit covering the visible to long-wave infrared parts of the spectrum. The fourth build in the series, set to be launched on the Joint Polar-orbiting Satellite System 3 (JPSS-3) platform, has recently completed its main ground calibration program and is set to be integrated into the satellite bus in the near future. This calibration program covered a comprehensive series of performance metrics designed to demonstrate the quality of the science data and ensure the instrument can maintain its calibration successfully once on-orbit. The subject of this work covers the radiometric calibration metrics including dynamic range, signal-to-noise ratio/noise equivalent differential temperature (SNR/NEdT), polarization sensitivity, scattered light response, relative spectral response, response versus scan angle, and uniformity, as well as uncertainties; all key metrics met or exceeded their design requirements with some minor exceptions. Comparisons to previous builds will also be provided.

JPSS-3↗

Metric analysis and data validation across FORTRAN projects

The desire to predict the effort in developing or explaining the quality of software has led to the proposal of several metrics. As a step toward validating these metrics, the Software Engineering Laboratory (SEL) has analyzed the software science metrics, cyclomatic complexity, and various standard program measures for their relation to effort (including design through acceptance testing), development errors (both discrete and weighted according to the amount of time to locate and fix), and one another. The data investigated are collected from a project FORTRAN environment and examined across several projects at once, within individual projects and by reporting accuracy checks demonstrating the need to validate a database. When the data comes from individual programmers or certain validated projects, the metrics' correlations with actual effort seem to be strongest. For modules developed entirely by individual programmers, the validity ratios induce a statistically significant ordering of several of the metrics' correlations. When comparing the strongest correlations, neither software science's E metric cyclomatic complexity not source lines of code appears to relate convincingly better with effort than the others.

Basili, Victor R.↗

Circularity Futures Workshop Series: Summary Report

The aim of this report is to synthesize key feedback received from the three-part Circularity Futures workshop series held in Spring 2024. The workshop series was conducted by the National Renewable Energy Laboratory (NREL) on behalf of U.S. Department of Energy, Office Energy Efficiency and Renewable Energy (EERE), and was broken into three workshops: Workshop 1 - Circularity Analysis Needs and Priorities; Workshop 2 - Circularity Metrics and Indicators; and Workshop 3 - Circularity Data. Together, the workshops focused on identifying the existing priorities and gaps in the circularity modeling space, understanding different stakeholders' use and interpretation of circularity metrics and indicators, identifying common data gaps and data quality challenges, and assessing the robustness of available solutions. The workshop series brought a diverse group of stakeholders - including representatives from U.S. government offices, national labs, nonprofit organizations, industry, and academia - to collect first-hand feedback on needs, priorities, challenges and opportunities in the circularity modeling and analysis space. The workshop discussions highlighted numerous common needs, priorities and challenges among the interviewed groups. Several topics were frequently discussed, including: 1) Circularity as a pathway for sustainable economic growth: While circularity is generally defined in terms of resource conservation and reducing wasteful disposal of materials, participants agreed that circular strategies should serve broader economic, environmental, and social goals. It is therefore crucial for circularity analysis to look beyond waste reduction and instead evaluate a variety of impact metrics such as cost savings, job creation, air quality, and pollutant emissions. Mutli-criteria decision-making frameworks may be useful for making sense of disparate metrics and evaluating tradeoffs between impact categories.; 2) Economic and social factors are not well understood: Underdevelopment of existing end-of-life (EOL) management infrastructure, inconsistent standardization codes and policy space in reusing recycled content, and suboptimal collection and sorting strategies collectively contribute to uncertainty about the economic potential of circular pathways. The latter observation is consistent among all technologies but more emphasized for renewable energy systems. Social impacts of circularity practices are less understood and less researched than other sustainability aspects.; 3) Inconsistent methods for assessing emerging technologies: LCA and TEA results vary widely depending on the assumptions made with regards to market adoption of new technologies. Emerging technologies suffer limited availability of data needed to conduct a robust circularity analysis. Yet, understanding projected impacts of proposed nascent technology is a key need for different stakeholder groups.; and 4) Lack of temporally and geospatially explicit data: There is a need for open data that represents variations in circularity technologies over time and location. The lack thereof leads to aggregated and potentially misrepresented results in circularity analysis. Sensitivity analyses should be included to verify whether options perceived as more sustainable align with real-world practices.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Hidden Connections between Regression Models of Strain-Gage Balance Calibration Data

Hidden connections between regression models of wind tunnel strain-gage balance calibration data are investigated. These connections become visible whenever balance calibration data is supplied in its design format and both the Iterative and Non-Iterative Method are used to process the data. First, it is shown how the regression coefficients of the fitted balance loads of a force balance can be approximated by using the corresponding regression coefficients of the fitted strain-gage outputs. Then, data from the manual calibration of the Ames MK40 six-component force balance is chosen to illustrate how estimates of the regression coefficients of the fitted balance loads can be obtained from the regression coefficients of the fitted strain-gage outputs. The study illustrates that load predictions obtained by applying the Iterative or the Non-Iterative Method originate from two related regression solutions of the balance calibration data as long as balance loads are given in the design format of the balance, gage outputs behave highly linear, strict statistical quality metrics are used to assess regression models of the data, and regression model term combinations of the fitted loads and gage outputs can be obtained by a simple variable exchange.

Ulbrich, Norbert↗

Uncertainty-Aware Machine Learning for Small-Angle X-ray Scattering Analysis in Autonomous Experimentation

Small-angle X-ray scattering (SAXS) is a powerful high-throughput characterization tool for probing nanoscale structure in native sample environments, providing real-time morphological information such as nanoparticle size and shape during synthesis. However, automated SAXS data analysis for extracting meaningful structural parameters is non-trivial and remains a bottleneck in closed-loop experimentation towards autonomous materials discovery, which demands fast, reliable, and uncertainty-aware data analysis. Here, we develop a machine-learning approach for automated SAXS analysis tailored to closed-loop nanoparticle synthesis. A Random Forest (RF) regression model is trained on 100,000 synthetic SAXS curves generated from polydisperse spherical nanoparticles with realistic background contributions. Using normalized one-dimensional SAXS intensity profiles as input, the RF model directly predicts nanoparticle radius, size polydispersity, and background parameters, while the ensemble standard deviation across trees provides built-in uncertainty quantification (UQ). On synthetic data, we show that combining fit-quality metrics (R 2 , MAE) with thresholds on prediction uncertainty reliably identifies accurate parameter estimates without access to ground truth. We then apply the trained model to 365 experimental SAXS profiles of citrate-reduced gold nanoparticles synthesized using an automated droplet-flow microreactor with in situ SAXS at a synchrotron beamline, classifying the results into high- and low-confidence subsets based on UQ metrics. Finally, we integrate RF-based SAXS analysis into a simulated closed-loop optimization campaign using Gaussian process Bayesian optimization to minimize nanoparticle polydispersity, benchmarking against conventional automated Levenberg–Marquardt fitting. The RF-guided campaign exhibits substantially faster convergence and lower relative opportunity cost (∼0.07 vs ∼0.3), demonstrating that uncertainty-aware machine-learning SAXS analysis significantly enhances the efficiency and robustness of autonomous nanomaterials synthesis workflows.

Bayesian optimization↗

Sensor Data Analytics and Data Quality Assessment Software

The proposed framework derives a set of quality metrics to provide critical insights into and tracking of grid operations, sensor performance, sensor longevity, and event statistics. Power grid engineers can utilize this information to identify problems with existing sensor locations and problematic power grid assets including generators, transmission lines, load centers, and substations. This information can also be used to identify unexpected/abnormal behavior of power grid components, improve power grid observability, and operational monitoring, and thus enhance real-time decision-making support system. Power grid planners can utilize this information to augment existing sensing architecture with new sensors and improve the observability of the network.

Mahapatra, Kaveri↗

Ubiquitous Traffic Volume Estimation through Machine-Learning Procedure

Traffic volume data is one of the most important metrics for accurate assessment of the performance of a transportation system. Quality volume data is required to effectively assess extent of delay and congestion, detect real-time perturbations to the network, and understand traffic patterns during major weather events. Traffic volume on freeways are typically collected through continuous count stations installed by state DOTs, while there is lack of traffic volume observability on off-freeway roads. The National Renewable Energy Laboratory (NREL), in Collaboration with the I-95 Corridor Coalition and the University of Maryland, extended its research into estimating volumes anywhere anytime from industry probe based data for off-freeway roads. NREL combined vehicle probe count data with several other data sets (speed, whether, roadway geometry, time-of-day, day-of-week, etc.) to estimate hourly volumes as well as AADTs. The research validated and demonstrated the machine learning model, namely XGBoost, using data collected from Pennsylvania, North Carolina, and Tennessee.

47 OTHER INSTRUMENTATION↗

Perovskite- and Dye-Sensitized Solar-Cell Device Databases Auto-generated Using ChemDataExtractor

The number of scientific publications reporting cutting-edge third-generation photovoltaic devices is increasing rapidly, owing to the pressing need to develop renewable-energy technologies that address the climate-change crisis. Consequently, the field could benefit from a central repository where photovoltaic-performance metrics, such as the power-conversion efficiency (η) are recorded. We present two automatically generated databases that contain photovoltaic properties and device material data for dye-sensitized solar cells (DSCs) and perovskite solar cells (PSCs), totalling 660,881 data entries representing 57,678 photovoltaic devices. The databases were generated by applying the text-mining toolkit ChemDataExtractor on a corpus of 25,720 articles. A multi-faceted evaluation, incorporating manual and automatic methods, was applied to ensure that the data contained therein were of the highest quality, with precision metrics ranging from 73.1% to 95.8%. The DSC database contains 475,045 entries representing 41,680 devices, and the PSC database contains 185,836 entries representing 15,818 devices. The databases are available in MongoDB and JSON formats, which can be queried in Python, R, Java and MATLAB for data-driven photovoltaic materials discovery.

14 SOLAR ENERGY↗

A Framework for Evaluating Climate Model Performance Metrics

The CMIP5 archive contains future climate projections from over 50 models provided by dozens of modeling centers from around the world. Individual model projections, however, are subject to biases created by structural model uncertainties. As a result, ensemble averaging of multiple models is often used to add value to model projections: consensus projections have been shown to consistently outperform individual models. Previous reports for the IPCC establish climate change projections based on an equal-weighted average of all model projections. However, certain models reproduce climate processes better than other models. Should models be weighted based on performance? Unequal ensemble averages have previously been constructed using a variety of mean state metrics. What metrics are most relevant for constraining future climate projections? This project develops a framework for systematically testing metrics in models to identify optimal metrics for unequal weighting multi-model ensembles. A unique aspect of this project is the construction and testing of climate process-based model evaluation metrics. A climate process-based metric is defined as a metric based on the relationship between two physically related climate variables?e.g., outgoing longwave radiation and surface temperature. Metrics are constructed using high-quality Earth radiation budget data from NASA's Clouds and Earth's Radiant Energy System (CERES) instrument and surface temperature data sets. It is found that regional values of tested quantities can vary significantly when comparing weighted and unweighted model ensembles. For example, one tested metric weights the ensemble by how well models reproduce the time-series probability distribution of the cloud forcing component of reflected shortwave radiation. The weighted ensemble for this metric indicates lower simulated precipitation (up to .7 mm/day) in tropical regions than the unweighted ensemble: since CMIP5 models have been shown to overproduce precipitation, this result could indicate that the metric is effective in identifying models which simulate more realistic precipitation. Ultimately, the goal of the framework is to identify performance metrics for advising better methods for ensemble averaging models and create better climate predictions.

Noel C Baker↗

Quality Ranking System for Molten Salt Thermal Property Data

A simple ranking system for use with data and property values compiled in the Molten Salt Thermal Database-Thermophysical Properties (MSTDB-TP) is presented. The approach includes assessments of five features of data collection: the method used to measure the property value, calibration of devices used to make the measurements, the quantified uncertainty of the results, control of the environmental conditions, and determination of the salt composition. The verifiability of how the property value was determined is also assessed based on information provided or cited within the data set. The quality of each facet is ranked as High, Moderate, or Incomplete based on objective metrics. The overall quality of the dataset is ranked based on rankings of the five data features and the reported property value, with rankings of A and B indicating the data and property value are deemed suitable for quantitative use and U indicating uncertainties that restrict the recommended use to qualitative purposes. The system is suitable for assessing data already in the MSTDB-TP and screening new entries.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Convective Weather Forecast Quality Metrics for Air Traffic Management Decision-Making

Since numerical weather prediction models are unable to accurately forecast the severity and the location of the storm cells several hours into the future when compared with observation data, there has been a growing interest in probabilistic description of convective weather. The classical approach for generating uncertainty bounds consists of integrating the state equations and covariance propagation equations forward in time. This step is readily recognized as the process update step of the Kalman Filter algorithm. The second well known method, known as the Monte Carlo method, consists of generating output samples by driving the forecast algorithm with input samples selected from distributions. The statistical properties of the distributions of the output samples are then used for defining the uncertainty bounds of the output variables. This method is computationally expensive for a complex model compared to the covariance propagation method. The main advantage of the Monte Carlo method is that a complex non-linear model can be easily handled. Recently, a few different methods for probabilistic forecasting have appeared in the literature. A method for computing probability of convection in a region using forecast data is described in Ref. 5. Probability at a grid location is computed as the fraction of grid points, within a box of specified dimensions around the grid location, with forecast convection precipitation exceeding a specified threshold. The main limitation of this method is that the results are dependent on the chosen dimensions of the box. The examples presented Ref. 5 show that this process is equivalent to low-pass filtering of the forecast data with a finite support spatial filter. References 6 and 7 describe the technique for computing percentage coverage within a 92 x 92 square-kilometer box and assigning the value to the center 4 x 4 square-kilometer box. This technique is same as that described in Ref. 5. Characterizing the forecast, following the process described in Refs. 5 through 7, in terms of percentage coverage or confidence level is notionally sound compared to characterizing in terms of probabilities because the probability of the forecast being correct can only be determined using actual observations. References 5 through 7 only use the forecast data and not the observations. The method for computing the probability of detection, false alarm ratio and several forecast quality metrics (Skill Scores) using both the forecast and observation data are given in Ref. 2. This paper extends the statistical verification method in Ref. 2 to determine co-occurrence probabilities. The method consists of computing the probability that a severe weather cell (grid location) is detected in the observation data in the neighborhood of the severe weather cell in the forecast data. Probabilities of occurrence at the grid location and in its neighborhood with higher severity, and with lower severity in the observation data compared to that in the forecast data are examined. The method proposed in Refs. 5 through 7 is used for computing the probability that a certain number of cells in the neighborhood of severe weather cells in the forecast data are seen as severe weather cells in the observation data. Finally, the probability of existence of gaps in the observation data in the neighborhood of severe weather cells in forecast data is computed. Gaps are defined as openings between severe weather cells through which an aircraft can safely fly to its intended destination. The rest of the paper is organized as follows. Section II summarizes the statistical verification method described in Ref. 2. The extension of this method for computing the co-occurrence probabilities in discussed in Section HI. Numerical examples using NCWF forecast data and NCWD observation data are presented in Section III to elucidate the characteristics of the co-occurrence probabilities. This section also discusses the procedure for computing throbabilities that the severity of convection in the observation data will be higher or lower in the neighborhood of grid locations compared to that indicated at the grid locations in the forecast data. The probability of coverage of neighborhood grid cells is also described via examples in this section. Section IV discusses the gap detection algorithm and presents a numerical example to illustrate the method. The locations of the detected gaps in the observation data are used along with the locations of convective weather cells in the forecast data to determine the probability of existence of gaps in the neighborhood of these cells. Finally, the paper is concluded in Section V.

Chatterji, Gano B.↗

Resiliency-Driven Proactive Distribution System Reconfiguration With Synchrophasor Data

Measuring and enabling resiliency of electric distribution systems with increasing weather and cyber events are important. Some of the extreme events (e.g. Earthquakes, Hurricanes) and associated paths are predicted and monitored closely in advance and allow to take pre-event proactive control actions. The Distribution Phasor Measurement Units (D-PMUs) provide new opportunities and supporting such proactive actions. Here, a synchrophasor based resiliency driven pre-event reconfiguration can ensure minimizing impact of the expected event on the power distribution system and associated performance. However, the D-PMUs will also face challenges in terms of data quality similar to the transmission PMUs. The focus of this paper is to provide data mining approaches for anomaly detection in D-PMUs and proposing resiliency-driven pre-event reconfiguration with islanding as a proactive mechanisms to minimize the impact of adverse events on system using processed synchrophasors data. Results are validated for real industrial feeders and test cases with satisfactory response

controlled islanding↗

Wind Plant Performance Prediction Benchmark Phase 1 (Technical Report)

Financial risk resulting from the uncertainty associated with developing, owning, and operating wind power plants remains a barrier to reducing the levelized cost of energy (LCOE). On average, modern wind power plants in the U.S. underperform their expected annual energy output by 3.5-4.5% , with many underperforming by over 10%. To compensate for this uncertainty, investors require a larger return on investment (ROI) and apply "knock-down" factors that mask much of the underlying sources of uncertainty. Wind energy projects thus have reduced access to low-cost capital. Furthermore, operating wind plants often take a simple approach to estimating operations & maintenance (O&M) costs (e.g. straight-line estimates based on similar plants), which can eat into profits. To overcome these issues, the wind industry must improve the models they use for estimating wind plant performance and operations. An industry consortium (IC) requested that the National Renewable Energy Laboratory (NREL) lead a Department of Energy (DOE) working group to benchmark the accuracy of wind power plant energy predictions against real operational data. The IC was also motivated by DOE and NREL's potential to characterize systematic energy underperformance, identify sources of uncertainty, and explore root causes. The Wind Plant Performance Prediction (WP3) project was created out of this request, and this report represents the successful completion of Phase 1 of the WP3 project. During the project, wind plant owners provided both pre-construction and operational data to NREL. The pre-construction data was provided to wind resource assessment (WRA) consultants so they could conduct energy yield assessments (EYA). NREL took all of the completed EYAs, along with the operational data, and conducted an operational assessment to benchmark the EYA results against actual operational data. Given the large amounts of sensitive data required for this effort, as well as historical opposition to sharing data within industry, successful completion of Phase 1 represents an unprecedented milestone for industry data sharing. To improve the accuracy and confidence of pre-construction EYAs, wind plant owners and investors need better, more certain, energy yield predictions. The WP3 Benchmark Project is an industry-driven response to this reality. For the first time, industry has taken the important step of working together at scale, sharing valuable operational data with DOE and NREL in order to investigate the sources of bias and uncertainty in these energy estimates. This IC provides wind plant preconstruction and operational data to NREL in an organized and documented fashion and provides guidance and feedback as needed. The IC also provides introspection of the design of experiment, key metrics of success, data challenges, analysis best practices, and quality of results.

17 WIND ENERGY↗