Search NASA⌕ Search

SEARCH · Search NASA

Results for “User evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Evaluating User Errors and Temporal Trends in Marine Fish Communities Using 360-Degree Underwater Photography

The use of environmental DNA (eDNA) sampling has been proposed as a complementary method to monitor fish species in marine environments, offering a non-invasive and potentially more efficient approach to marine species observations. eDNA monitoring could be especially useful in and around sites targeted for marine energy generation as these regions need regular monitoring that would be impractical with traditional techniques. Before we can fully rely upon eDNA, we must first verify its accuracy against other proven methods, such as the use of underwater photography. In this study, I deployed a 360-degree camera in the tidal channel of Sequim Bay once a month during several hours overlapping slack tide. I investigated how having multiple people identify and count fish on underwater images could affect the overall results. Using chi square tests in R, I compared my fish identifications and counts to those made by another intern on the same images recorded in August. I found significant differences in the number of species identified and the total individual counts between the two different datasets. I also tested the statistical differences in both Shannon diversity and Pielou evenness indices between the August, September, and November camera deployments using a Hutcheson t-test. Only one significant difference was found in the Shannon index comparisons, and none were found between the Pielou evenness comparisons. These findings show that if multiple identifiers are used to process underwater images, quality control checks must be made to reduce the potential for error. This also points toward the possibility to leverage more advanced image analysis processes, such as automated image analysis software. The findings from this study also show that the dynamics of marine fish communities can vary over a few months; however, further analysis is needed to determine the extent of the seasonal changes in Sequim Bay.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Visualization Quality Assessment

Understanding how inaccuracies in visualizations affect users’ perception and understanding of scientific data is hard. Inaccuracies in visualizations are quite common and could arise from a range of sources such as errors in the original dataset arising from compression artifacts, errors in the capturing device, noise during transmission of the data, effects due to the algorithm being used to convert data to visualization images, images generated from neural networks, and sources we have yet to discover. Many image quality assessment metrics have been developed to quantify image errors. However, these are usually focused on “natural images” rather than visualizations of scientific data. Common image quality assessment metrics (IQAs) include MSE, PSNR, perceptual metrics such SSIM, FSIM as well as perceptual metrics using deep learning approaches. However, a critical part of understanding how errors are perceived by humans, and subsequently developing more accurate quality assessment metrics, is through user evaluation studies. The goal of this software is to develop a visualization quality assessment (VQA) process that will enable the generation of VQAs that can be used to quantify errors in scientific data visualizations. The VQA development process will include software to support user evaluation experimental design, analysis of visualization differences against standard quality metrics, and the ability to develop additional VQA metrics specific to scientific visualization images.

Grosset, Andre↗
DOE OSTI2026

CO2 Handling & Electrolyzer Efficiency Scaling Evaluator

CHEESE is an interactive dashboard tool for estimating the performance and material requirements of carbon dioxide electrolysis systems. It helps users evaluate how electrode area, current density, product selectivity, gas flow, cell voltage, and the number of cells in a stack affect system operation. The dashboard provides simple and advanced modes so it can be used for both quick estimates and more detailed engineering analysis. Users can estimate product output, carbon dioxide use, electrical power, electrode area, gas and liquid flow rates, energy efficiency, material cost per test, and the effect of scaling from a single cell to a multi cell stack. It also includes tools for examining carbon balance, equipment durability, component replacement, and changes in performance over time. This is intended to help both academia and industry researchers who are either getting into CO2 electrolysis on lab-scale or are trying to establish a larger footprint. CHEESE presents results through tables, charts, and simplified cell and stack diagrams. It is intended to support research planning, experimental design, comparison of operating conditions, and early stage scale up studies.

Prajapati, Aditya [Lawrence Livermore National Lab↗
DOE OSTI2025

Developing Digital Twin Visualizations: A Methodology and Case Study on Chemical Separation Processing

As advances in digital engineering continue to push the technological boundaries, digital twin (DT) visualizations for diagnostics and safeguards advancement become much more feasible and practical. DTs generate large and complex data streams that require effective user interfaces to provide monitoring and diagnostic capabilities. Unfortunately, while these frameworks exist, there is not much research on the systematic documentation of human–computer interaction (HCI) for DT visualization. This work presents a dual-mode visualization methodology (two dimensional [2D] graphical user interface dashboard and 3D mixed reality) designed to support diagnostic tasks in DT systems and building on a validated framework and applying established HCI principles. The methodology is demonstrated through a case study of aqueous processing at Idaho National Laboratory, using experimental data from the chemical solvent extraction runs. Our interfaces display real-time alerts and monitoring to inform users of safeguards anomalies. The interfaces use immersive 3D mixed-reality visualization for further system and experiment investigation. This work demonstrates how the systematic application of HCI principles can inform DT visualization design for diagnostic and safeguards applications. While formal user evaluation studies remain as future work, this paper documents the systematic design methodology and demonstrates a proof-of-concept implementation.

3D visualization↗
DOE OSTI2025

“Which Projections Do I Use?” Strategies for Climate Model Ensemble Subset Selection Based on Regional Stakeholder Needs

Climate model (or earth system model) projections are increasingly used for climate adaptation planning and impact assessments. As part of this process, many end‐users evaluate a subset of downscaled climate projections without being aware of the implications of downscaling methodology for statistics or event outcomes. Approaches for determining a subset of global climate models to use often focus on values from the raw models, rather than from their downscaled counterparts, in other words assuming that the statistical distribution of the multi‐model ensemble does not change post downscaling. This study demonstrates that a downscaled ensemble will typically retain the change distribution as a raw ensemble, but individual models can differ dramatically post‐downscaling. We recommend that subset‐selection methods account for this possibility and that decision‐relevant downscaled climate projections provide proper descriptions of fitness‐for‐purpose and essential caveats, so that non‐specialists can interpret the results with an appropriate level of confidence.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Community Public Mobility Using On-Demand, Low-Speed Electric Vehicles: A Case Study in Downtown St. Louis, Missouri

Legacy fixed route transit systems designed to serve commuters struggle to provide efficient and effective service for short neighborhood trips and for population groups unable to access and egress transit stops using active modes (e.g., elderly, disabled). Neighborhood on-demand transit (ODT) services using low-speed electric vehicles (LSEV) are an innovative technological solution that can help fill this gap in service (e.g., short, high-frequency trips) for diverse populations and trip types. This study evaluated user characteristics and travel behavior for a neighborhood ODT service (using LSEVs) in downtown St. Louis, Missouri using responses from a community survey (n=244), ridership data, and vehicle trajectory information. A comparative analysis between neighborhood ODT, fixed route transit, and transportation network companies (TNC) was also conducted from the perspectives of total travel time, cost, and greenhouse gas emissions. Ultimately, the goal of the analysis was to motivate and inform holistic public mobility systems where different services are optimized to meet specific community needs. Findings indicate that the neighborhood ODT was effective at reaching diverse populations (elderly (20%), lower income (27%), and households with limited access to private vehicles (34%)). ODT reduced total travel time by 32% compared to fixed route transit, produced 2.4 - 4.3 times less greenhouse gas emissions per passenger mile (compared to transit and TNCs), and was more affordable (free to users) than alternative options ($1 for transit, $10-12 for TNCs). Overall satisfaction rates were high, with 80% of respondents rating the service a 4 or 5 out of 5.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

REopt Lite Tutorial: International Locations

NREL’s REopt Lite web tool helps users evaluate the economic viability of distributed photovoltaic (PV), wind, battery storage, combined heat and power, and thermal energy storage systems. REopt Lite was designed for locations within the United States; however, with appropriate adjustments, it is possible to use most of its features for international locations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Community Public Mobility Using On-Demand, Low-Speed Electric Vehicles: A Case Study in Downtown St. Louis, Missouri

Legacy fixed route transit systems designed to serve commuters struggle to provide efficient and effective service for short neighborhood trips and for population groups unable to access and egress transit stops using active modes (e.g., elderly, disabled). Neighborhood on-demand transit (ODT) services using low-speed electric vehicles (LSEV) are an innovative technological solution that can help fill this gap in service (e.g., short, high-frequency trips) for diverse populations and trip types. This study evaluated user characteristics and travel behavior for a neighborhood ODT service (using LSEVs) in downtown St. Louis, Missouri using responses from a community survey (n=244), ridership data, and vehicle trajectory information. A comparative analysis between neighborhood ODT, fixed route transit, and transportation network companies (TNC) was also conducted from the perspectives of total travel time, cost, and greenhouse gas emissions. Ultimately, the goal of the analysis was to motivate and inform holistic public mobility systems where different services are optimized to meet specific community needs. Findings indicate that the neighborhood ODT was effective at reaching diverse populations (elderly (20%), lower income (27%), and households with limited access to private vehicles (34%)). ODT reduced total travel time by 32% compared to fixed route transit, produced 2.4 - 4.3 times less greenhouse gas emissions per passenger mile (compared to transit and TNCs), and was more affordable (free to users) than alternative options ($1 for transit, $10-12 for TNCs). Overall satisfaction rates were high, with 80% of respondents rating the service a 4 or 5 out of 5.

ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

Hydropower Biological Evaluation Toolset (HBET) Version 3.0: User Guide

The Hydropower Biological Evaluation Tools (HBET) software package, developed by Pacific Northwest National Laboratory (PNNL), is designed to assemble, organize, and process data collected by Sensor Fish and live fish. HBET enable users to characterize the hydraulic conditions of hydropower structures and estimate fish injury and mortality rates from various stressors. Future updates of the software may support other technologies, such as bead tracking in physical models and computational fluid dynamics. The HBET program can be customized to analyze different hydraulic applications, including turbines, spillways, weirs, pumped storage, and other user-defined functions, and therefore, help researchers, turbine designers, hydropower operators, and regulators better evaluate hydropower structures regarding their environmental sustainability and cost-effectiveness. Added content to the user guide about the new feature for predicting absolute injury rates.

13 HYDRO ENERGY↗
DOE OSTI2020

Feasibility Analysis of Converter-Interfaced Combined Heat and Power System

As a promising new design concept, the converter-interfaced combined heat and power (CHP) system is coupled to the bulk grid through a rectifier and a grid-ready tied inverter. Compared to the traditional directly-coupled CHP system, it removes the requirement for oversizing the CHP generator, limits the short-circuit contribution of the generator and simplifies the grid integration process of CHP system. This paper evaluates the economic benefits of this concept by calculating the annualized Return-on-Investment (ROI) and comparing it to the directly-coupled system. The economic analysis includes timeseries simulations to compute energy transactions with the bulk grid as well as sizing the equipment to calculate the capital and operational costs. Obtained results indicate that in majority of user cases evaluated, the converter-interfaced CHP systems can provide better ROI than directly-coupled systems. Given the additional technical benefits provided by inverter-based distributed energy resources (DERs), the proposed concept is proved to be technically viable and economically feasible.

03 NATURAL GAS↗
DOE OSTI2024

Operational Analytics Studies for ATLAS Distributed Computing: Data Popularity Forecast and Utilization of the WLCG Centers

Operational analytics is the direction of research related to the analysis of the current state of computing processes and the prediction of future states in order to anticipate imbalances and take timely measures to stabilize a complex system. There are two relevant areas in ATLAS Distributed Computing that are currently the focus of studies: user physics analysis including the forecast of popularity of data samples among users, and evaluating WLCG centers for their readiness to process user analysis payloads. Studying these areas is challenging due to the complexity involved, as it requires a comprehensive understanding of numerous boundary conditions typically found in large-scale distributed computing infrastructures. Forecasts of data popularity are problematic without the categorization of user tasks by their types (data transformation or physics analysis), which do not always appear on the surface but may induce noise, which introduces significant distortions for predictive analysis. Evaluating the WLCG resources by their analysis workloads is also a challenging task as it is necessary to find a balance between the workload of the resource, its performance, the waiting time for jobs on it, as well as the volume of jobs that it processes. This is especially difficult in a heterogeneous computing environment, where legacy resources are used along with modern high-performance machines. We will look at these areas of research in detail and discuss what tools and methods are used in our work, demonstrating results already obtained.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

HydrogenAR: Interactive Data-Driven Presentation of Dispenser Reliability: Preprint

When delivering presentations to a co-located audience, we typically use slides with text and 2D graphics to complement the spoken narrative. Though presentations have largely been explored on 2D media, augmented reality (AR) allows presentation designers to add data and augmentations to existing physical infrastructure on display. This could provide a more engaging experience to the audience and support comprehension. With HydrogenAR, we present a novel application that leverages the benefits of data-driven storytelling with those of AR to explain the unique challenges of hydrogen dis- penser reliability. Utilizing physical props, situated data, and virtual augmentations and animations, HydrogenAR serves as a unique presentation tool, particularly critical for stakeholders, tour groups, and VIPs. HydrogenAR is a product of multiple collaborative de- sign iterations with a local Hydrogen Fuel research team and is evaluated through interviews with team members and a user study with end-users to evaluate the usability and quality of the interactive AR experience. Through this work, we provide design considera- tions for AR data-driven presentations and discuss how AR could be used for innovative content delivery beyond traditional slide-based presentations.

augmented reality↗
DOE OSTI2022

Robust High-Throughput Phenotyping with Deep Segmentation Enabled by a Web-Based Annotator

The abilities of plant biologists and breeders to characterize the genetic basis of physiological traits are limited by their abilities to obtain quantitative data representing precise details of trait variation, and particularly to collect this data at a high-throughput scale with low cost. Although deep learning methods have demonstrated unprecedented potential to automate plant phenotyping, these methods commonly rely on large training sets that can be time-consuming to generate. Intelligent algorithms have therefore been proposed to enhance the productivity of these annotations and reduce human efforts. We propose a high-throughput phenotyping system which features a Graphical User Interface (GUI) and a novel interactive segmentation algorithm: Semantic-Guided Interactive Object Segmentation (SGIOS). By providing a user-friendly interface and intelligent assistance with annotation, this system offers potential to streamline and accelerate the generation of training sets, reducing the effort required by the user. Our evaluation shows that our proposed SGIOS model requires fewer user inputs compared to the state-of-art models for interactive segmentation. As a case study of the use of the GUI applied for genetic discovery in plants, we present an example of results from a preliminary genome-wide association study (GWAS) of in planta regeneration in Populus trichocarpa (poplar). We further demonstrate that the inclusion of a semantic prior map with SGIOS can accelerate the training process for future GWAS, using a sample of a dataset extracted from a poplar GWAS of in vitro regeneration. The capabilities of our phenotyping system surpass those of unassisted humans to rapidly and precisely phenotype our traits of interest. The scalability of this system enables large-scale phenomic screens that would otherwise be time-prohibitive, thereby providing increased power for GWAS, mutant screens, and other studies relying on large sample sizes to characterize the genetic basis of trait variation. Our user-friendly system can be used by researchers lacking a computational background, thus helping to democratize the use of deep segmentation as a tool for plant phenotyping.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Strategic Performance Evaluation Measurement Plan (PEMP) Scorecard in QuickScore Training Tool (User Guide)

The Strategic Performance Evaluation Measurement Plan (PEMP) Scorecard, now housed in QuickScore, is an assurance and governance data scorecard which shows the health of the labs against the current fiscal year PEMP objectives. POC's for PEMP objective owners are requested to provide status of their PEMP objective, capture the top 6 cumulative accomplishments, top 6 issues, which include IFR issues and provide mitigation and or improvement action plans. Updated scorecards are used in OMR and BOM GS&S committee

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2022

Case Study: Applying the INL Resilience Framework to Iowa Lakes Electric Cooperative Distributed Wind Systems

Traditional metrics and evaluation methods for resiliency are not sufficient to evaluate the effect that distributed wind systems will have, particularly in light of the challenges described above. While the concept of resiliency is not new, its application to the electric grid is neither standardized nor well-defined, and there is little to no guidance on how to evaluate resilience specifically for distributed wind systems. To fill this gap, the Idaho National Laboratory (INL), as part of the multi-laboratory Microgrids, Infrastructure Resilience, and Advanced Controls Launchpad (MIRACL) project, has developed a resilience framework for electric energy delivery systems (EEDS). The framework provides detailed steps for evaluating resiliency in the planning, operational, and future stages, and encompasses five core functions of resilience. It allows users to evaluate the resilience of distributed wind, taking into consideration the resilience of the wind systems themselves, as well as the effect they have on the resiliency of any systems they are connected to. In this study, we evaluate the resilience of the distributed wind systems at Iowa Lakes Electric Cooperative to cybersecurity hazards. We show that the wind resource can benefit the overall system resilience during some hazards. We show that the practices in place make the wind subsystems resilient against some cybersecurity hazards but that there are still significant risks associated with other cybersecurity hazards

17 WIND ENERGY↗
DOE OSTI2021

Towards Semantic Search in Building Sensor Data

This paper presents a search engine system for sensor time series data and metadata in the context of building management. It takes natural language queries as input, retrieves sensor time series data, ranks them with respect to their relevance to a given query, and visualizes the time series as search results. In addition, the system allows users to interact with the search results: they can define events of interest in the visualized results and search across sensor data for similar events, i.e., the search by example scheme. Quantitative evaluations and user studies demonstrate the value of this system for managing building sensor data.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗