Search NASA⌕ Search

SEARCH · Search NASA

Results for “Usability evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Usability Evaluation of the Innovation Portal and Integrated Capability Analysis Platform

The usability evaluation of the ICAP and IP tools was conducted as part of the United States (U.S.) Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) ION effort with two evaluators for the Idaho National Laboratory. Many nuclear operations in the United States are facing challenges of increased business, operations, and maintenance costs due to subsidized renewables and natural gas. The commercial NPP industry has looked to other industries to develop methods such as Integrated Operations (IO) to use available technologies to reduce operating costs. One concept for reducing costs that has been developed is the Integrated Operations for Nuclear (ION). A capability Stack model was developed as part of the ION effort to break down information related to People, Technology, Processes, and Governance (PTPG). The Innovation Portal (IP) and Integrated Capability Analysis Platform (ICAP) were created from the Capability Stack model as part of the ION effort to help utilities reduce O&M costs by identifying work reduction opportunities. Two heuristic evaluations and a cognitive walkthrough of the ICAP and IP tools were completed to identify usability issues to further develop the tools for use in the ION effort. Usability issues identified were rated by priority levels and across both tools. The priority levels identified were high (5), medium (11), and low (10). Recommendation were made to help reduce usability issues of the systems and help increase the user response rate of using the systems once they are implemented for utilities. Future work on designs for the IP tool and additional heuristic evaluations and external testing are being developed to continue refinement of the ICAP and IP tools for the ION effort.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2022

Operator Insights and Usability Evaluation of Machine Learning Assistance for Power Grid Contingency Analysis

Introducing machine learning (ML) assistance into any established process comes with adoption barriers, including entrenched procedures, technological and human readiness levels, human-machine trust, and work culture resistance to change. These barriers are even greater in critical operations such as operating a national or regional power grid, in which both regulatory frameworks and the importance of maintaining reliability levels causes additional resistance to the adoption of new computational support. Developers of future systems and job aides must consider not only technical aspects, but also whether new systems are usable by power system operators. This work presents the methodology and results of a study to evaluate the usability and readiness of a prototype recommender system for power grid contingency analysis. We explore operator cognitive load and evaluate operator performance when solving a collection of scenarios both with and without recommender assistance. We also examine operator trust in the system. We report insights gained on the readiness of the system using a collection of evaluation techniques.

Human-Machine Teaming, Power Systems, usability ev↗
DOE OSTI2024

Modeling the impact of extreme summer drought on conventional and renewable generation capacity: Methods and a case study on the Eastern U.S. power system

Across recent years, there has been a growing prevalence of extreme weather events throughout the United States, posing significant challenges to the reliable and resilient operation of power systems. Specifically, summer droughts threaten to severely reduce available generation capacity to meet regional electricity demand, potentially leading to power outages. This underscores the importance of accurate resource adequacy (RA) assessment to ensure the reliable operation of the nation’s energy infrastructure. Accurately evaluating the usable capacity of regional generation fleets is a challenging undertaking due to the intricate interactions between power systems and hydro-climatic systems. Here, this paper proposes a systematic and analytical framework to evaluate the impacts of extreme summer drought events on the available capacity of various generating technologies, incorporating both meteorological and hydrologic factors. The framework provides detailed plant-level capacity derating models for hydroelectric, thermoelectric, and renewable power plants, facilitating evaluations with high temporal and spatial resolution. The application of the proposed impact assessment framework to the 2025 generation fleet of the real-world power system within the PJM and SERC regions of the United States yields insightful results. By analyzing the daily usable capacity of 6,055 at-risk generators across the study region, it shows that the summer capacity deration is most significant for hydroelectric and once-through thermal power plants, followed by recirculating thermal power plants and combustion turbines. In the event of the recurrence of the 2007 southeastern summer drought event in the near future, the generation fleet could experience a substantial reduction in available capacity, estimated at approximately 8.5 GW, compared to typical summer conditions. The sensitivity analysis reveals that the usable capacity of the generation fleet would suffer an even more significant decrease under conditions of increasingly severe summer droughts. The proposed approach and the findings of this study provide valuable methodologies and insights, empowering stakeholders to bolster the resilience of power systems against the potentially devastating effects of future extreme drought events.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Reliability of Open Public Electric Vehicle Direct Current Fast Chargers

The aim was to systematically evaluate the usability of all public electric vehicles (EV) direct current fast chargers (DCFC) in the San Francisco region. To achieve a rapid transition to EVs, a highly reliable and easy to use charging infrastructure is critical to building confidence among consumers. The functionality and usability of all 182 open, public DCFC charging stations with CCS connectors (combined charging system) in the 9 counties of the Bay Area were tested (655 electric vehicle service equipment (EVSE) ports). An EVSE was classified as functional if it charged an EV for 2 minutes. Overall, 73.3% of the 655 EVSEs were functional. The causes of the nonfunctioning EVSEs (23.5%) were blank or unresponsive screens or error messages; payment system failures; charge initiation failures; network failures; or broken connectors. In addition, the cable was too short to reach the EV inlet for 3.2% of the EVSEs. A random sampling of 10% of the EVSEs, approximately 8 days after the first evaluation, found no overall change in functionality. The level of functionality found with field testing conflicts with the 95–98% uptime reported by the EV service providers (EVSPs) who operate the EV charging stations. There is a need for precise and verifiable definitions of uptime, downtime, and excluded time, as applied to public EV chargers. In conclusion, the level of failure of the existing public EV DCFC charge infrastructure highlights the importance of improving the system design and maintenance to improve adoption of EVs.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2021

Examining the Use of the Technology Acceptance Model for Adoption of Advanced Digital Technologies in Nuclear Power Plants

The United States nuclear industry is being economically challenged and needs wide-scale adoption of enabling technology like advanced automation to remain viable. To enable wide-scale adoption of advanced technology, addressing the human-technology integration challenges and factors that influence technology acceptance (i.e., ‘buy in’) is pertinent. This work examines the Technology Acceptance Model as a framework for characterizing the factors influencing technology adoption in the nuclear industry. An outcome of this work is to support the development of future tools such as a technology acceptance survey that can be used to assess technology acceptance throughout the lifespan of a modernization program to provide a systematic way of ensuring stakeholder and operations ‘buy in.’

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

HydrogenAR: Interactive Data-Driven Presentation of Dispenser Reliability: Preprint

When delivering presentations to a co-located audience, we typically use slides with text and 2D graphics to complement the spoken narrative. Though presentations have largely been explored on 2D media, augmented reality (AR) allows presentation designers to add data and augmentations to existing physical infrastructure on display. This could provide a more engaging experience to the audience and support comprehension. With HydrogenAR, we present a novel application that leverages the benefits of data-driven storytelling with those of AR to explain the unique challenges of hydrogen dis- penser reliability. Utilizing physical props, situated data, and virtual augmentations and animations, HydrogenAR serves as a unique presentation tool, particularly critical for stakeholders, tour groups, and VIPs. HydrogenAR is a product of multiple collaborative de- sign iterations with a local Hydrogen Fuel research team and is evaluated through interviews with team members and a user study with end-users to evaluate the usability and quality of the interactive AR experience. Through this work, we provide design considera- tions for AR data-driven presentations and discuss how AR could be used for innovative content delivery beyond traditional slide-based presentations.

augmented reality↗
DOE OSTI2024

Assessment of Electric Grid Transmission System Simulator for Human Factors Research

Over the past decades, various technologies have been developed for electric grid operations to support clean energy, meet rising electricity demands, and address infrastructure concerns. However, the human factors aspect is often overlooked during rapid integration. Questions persist about how these technologies impact human performance. Simulators play a critical role in supporting investigation of human factors design concepts and conducting comprehensive usability testing to evaluate human performance and assess human reliability. This paper aims to address human factors research simulator requirements and conduct a comparative study of six different simulators. A detailed evaluation reveals that the evaluated simulators lack the ability to customize user interfaces. Additionally, their user interface designs do not fulfill the basic human factors design principles, potentially leading to increased response variability and reduced statistical power when conducting controlled experimental research. In the future, it is essential to develop scripting tools to integrate customizable user interfaces and simulation models, ensuring meeting research requirements.

Li, Ruixuan↗
DOE OSTI2020

An Integrated Energy Systems Prototype Human-System Interface for a Steam Extraction Loop System to Support Joint Electricity-Hydrogen Flexible Operations

Due to increasing economic competition from renewables and combined-cycle natural gas plants, nuclear power plants are looking toward flexible operations to enhance their cost competitiveness. The Integrated Energy Systems project under the Light Water Reactor Sustainability program of the U.S. Department of Energy focuses on joint electricity-hydrogen flexible operations. Joint electricity-hydrogen flexible operations entail the nuclear power plant diverting thermal energy via main steam to a hydrogen production plant located nearby. The steam serves to enhance the efficiency of the hydrogen production. Furthermore, high temperature electrolysis requires a large amount of electricity, which the plant can also provide. The plant provides steam and electricity to the hydrogen plant throughout the day, but during peak demand hours the nuclear power plant returns to solely providing electricity to meet the high demand. Through this flexible concept of operations, the plant can optimize the thermal energy it produces without having to maneuver the power of the plant. This report documents the human factors process to design and develop a prototype human-system interface for the steam extraction loop that serves as the conduit between the nuclear power plant and the adjacent hydrogen plant. The design process followed the human factors guidelines set by NUREG-0711, Human Factors Engineering Program Review Model (O’Hara, Higgins, & Fleger, 2012), and expanded upon by the Guideline for Operational Nuclear Usability and Knowledge Elicitation (GONUKE; Boring, Ulrich, Joe, & Lew, 2015; Boring, Lew, & Ulrich, 2016). The design process entailed operator interviews to determine the concept of operations for the steam extraction loop, a review and adaptation of digital interface design concepts developed by the team in prior projects, an iterative design process, and reviews conducted by both operators and human factors experts. Several versions of the prototype human-system interface were developed. Operators were interviewed to determine what design features they found useful and would like to see in the interface. The design underwent a review by human factors experts against NUREG-0700, Human Interface Design Review Guidelines (U.S. Nuclear Regulatory Commission, 2019), to ensure compliance with the latest human factors standards for digital interfaces in nuclear applications. The design was then prototyped as a functional windows-based application integrated with the Generic Pressurized Water Reactor simulator modified to include the steam extraction loop. The simulation is supported by the Human Systems Simulation Laboratory at Idaho National Laboratory, which supports operator-in-the-loop testing. This is an ongoing project and the next phase of the project entails performing an operator-in-the-loop usability study to evaluate the interface and examine the proposed concept of operations to extraction steam from the nuclear power plant for delivery to the coupled hydrogen production plant.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2025

Advancing material modeling in hydrocodes using a concurrent finite-element and molecular dynamics multiscale framework

We present a multiscale simulation framework that couples the finite-element method with molecular dynamics. Bypassing traditional equations of state (EOS) by using in-line atomistic simulations, the method offers the advantage of incorporating detailed microscale physics not easily represented with coarse-grained models. Coupling consistency with the continuum code is ensured through the use of lifting and restriction operators, in line with heterogeneous multiscale methods. The concurrent continuum-atomistic framework is validated through comparison with experimental results and conventional EOS models, and demonstrated in a shock-driven hydrodynamic flow simulation under extreme conditions. We further evaluate the framework's usability by comparing it to state-of-the-art EOS models of deuterium. A computational performance study reveals that the atomistic EOS evaluation is a feasible alternative to conventional approaches, and demonstrates a weak scaling of 99% efficiency. These results highlight the framework's potential for large-scale multiscale modeling across a broad range of materials and conditions.

Computer science↗
DOE OSTI2024

Distributed Neural Representation for Reactive In Situ Visualization

Implicit neural representations (INRs) have emerged as a powerful tool for compressing large-scale volume data. This opens up new possibilities for in situ visualization. However, the efficient application of INRs to distributed data remains an underexplored area. Here, in this work, we develop a distributed volumetric neural representation and optimize it for in situ visualization. Our technique eliminates data exchanges between processes, achieving state-of-the-art compression speed, quality and ratios. Our technique also enables the implementation of an efficient strategy for caching large-scale simulation data in high temporal frequencies, further facilitating the use of reactive in situ visualization in a wider range of scientific problems. We integrate this system with the Ascent infrastructure and evaluate its performance and usability using real-world simulations.

Wu, Qi↗
DOE OSTI2026

Level-2 Milestone 9009: Flux and Rabbit Capabilities on El Capitan

This document is the milestone delivery report for the ASC 2025 L2 milestone (See Table 1) for advanced I/O capabilities for El Capitan via Flux Workload Manager support and the new I/O hardware designed for El Capitan, the Rabbit Storage System. In this document we describe the design of the Rabbit Storage System and how it is managed by Flux. We evaluate the performance and usability of Rabbit using ARES, IOR, and an AI inference workload. Overall, we find that Rabbit shows good scalability, especially in node-local storage configurations, and is more scalable than the global Lustre parallel file system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Physical and Visual Evaluation of Filters for Direct Observation of the Sun and the International Standard ISO 12312-2:2015

We investigated the compliance of 43 commercially available solar filters (eclipse glasses) with the ISO 12312-2:2015 standard by measuring their spectral transmittances (280–2000 nm) and calculating their luminous, solar ultraviolet A, ultraviolet B, and infrared (IR) transmittances. We also evaluated the filters for usability by observing the full midday Sun and rating the view on a seven-point balanced scale, from “far too dark, details seen only with great difficulty” to “far too light, uncomfortable to view the Sun.” The mean ratings of two observers, one experienced and one inexperienced in solar observing, differed by 0.28 (95% confidence interval of the mean = 0.26). The inexperienced observer tended to be less accepting of high transmittances. All 43 solar filters complied with the UV and IR requirements. Eighteen filters passed the luminous transmittance requirements, and 24 were borderline too light or too dark. Seven of the 15 solar filters with a luminous transmittance darker than the requirement were rated as acceptable. One filter that passed and another that was borderline too light were rated as too light or far too light. The ISO 12312-2 limits derive from welding filter standards and do not represent an appropriate evidence base for direct solar viewing. This work provides the evidence base for a maximum 0.0012% and a minimum 0.00004% luminous transmittance for solar filters. The results of this study also support the use of welding filters between shades 12 and 16. Lighter welding filters are more acceptable than solar filters of the same luminous transmittance.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Improving the User Interface of the DeepLynx Data Warehouse

DeepLynx is an open-source ontology-based data warehouse created by INL to support the creation and life cycle of digital engineering projects, with a particular emphasis on digital twins [1]. Digital twins are systems that represent physical assets and process in a real-time digital environment [1]. Most well-known commercial data warehouses use Graphical User Interfaces (GUIs) for users to interact with their systems [3]. Limited publications have addressed the design of these interfaces and understanding of their target users. The current users and development team acknowledge the need to improve the current UI, not just for aesthetics but to improve functionality and workflow of DeepLynx. Traditional data warehouse users are developers, data scientists and business analysts [2]. DeepLynx users have a vast range of experience using data warehouses, and diverse roles, including engineers, scientists and management positions. Because there is a broader audience of target users for DeepLynx than a typical data warehouse, it is essential that DeepLynx has a useable and intuitive user interface. To achieve this the team performed human-computer interaction methods, including a Heuristic Evaluation of current UI using Neilsen’s Usability Heuristic, create personas based on current users by designing a user survey, data analysis and develop of personas. Followed by a redesign of the UI following using Neilsen’s Usability Heuristic and Norman’s Principles of Interactive Design in industry standard software Figma. Lastly a Heuristic Evaluation of new UI design, using Neilsen’s Usability Heuristic and User testing of redesign UI and have a group of users complete a Thinking Aloud Test of the new UI. Preliminary results of the Heuristic Evaluation of current UI arise issue with Consistency and Standards, Visibility of System Status, Match System and Real World and Recognition Rather than Recall. These issues were addressed in the proposed redesign by applying Neilsen’s Usability Heuristic and Norman’s Principles of Interactive Design. Next steps include formalized list of lessons learned and design implications for future publications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Metrics as tools for bridging climate science and applications

In climate science and applications, the term “metric” is used to describe the distillation of complex, multifaceted evaluations to summarize the overall quality of a model simulation, or other data product, and/or as a means to quantify some response to climate change. Metrics provide insights into the fidelity of processes and outcomes from climate models and can assist with both differentiating models' representation of variables or processes and informing whether models are “fit for purpose.” Metrics can also provide a valuable reference point for co-production of knowledge between climate scientists and climate impact practitioners. Although continued metric developments enable model developers to better understand the impacts of decisions made in the model design process, metrics also have implications for the characterization of uncertainty and facilitating analyses of underlying physical processes. As a result, comprehensive evaluation with multiple metrics enhances usability of climate information by both scientific and stakeholder communities. Here, this paper presents examples of insights gained from the development and appropriate use of metrics, and provides examples of how metrics can be used to engage with stakeholders and inform decision-making.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Visualizing the NIOSH Pocket Guide: Open-source web application for accessing and exploring the NIOSH Pocket Guide to Chemical Hazards

The NIOSH Pocket Guide to Chemical Hazards is a trusted resource that displays key information for a collection of chemicals commonly encountered in the workplace. Entries contain chemical structures—occupational exposure limit information ranging from limits based on full-shift time-weighted averages to acute limits such as short-term exposure limits and immediately dangerous to life or health values, as well as a variety of other data such as chemical-physical properties and symptoms of exposure. The NIOSH Pocket Guide (NPG) is available as a printed, hardcopy book, a PDF version, an electronic database, and a downloadable application for mobile phones. All formats of the NIOSH Pocket Guide allow users to access the data for each chemical separately, however, the guide does not support data analytics or visualization across chemicals. This project reformatted existing data in the NPG to make it searchable and compatible with exploration and analysis using a web application. The resulting application allows users to investigate the relationships between occupational exposure limits, the range and distribution of occupational exposure limits, and the specialized sorting of chemicals by health endpoint or to summarize information of particular interest. These tasks would have previously required manual extraction of the data and analysis. The usability of this application was evaluated among industrial hygienists and researchers and while the existing application seems most relevant to researchers, the open-source code and data are amenable to modification by users to increase customization.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

eCC++ : A Compiler Construction Framework for Embedded Domain-Specific Languages

eCC++ is a new compiler construction framework for embedding domain-specific programming languages within C++. That is, the host language is C++, the guest language is the DSL to be embedded in C++, and eCC++ is the tool that enables the embedding. The eCC++ framework is composed of three main components: a front-end, an API for program verification based on a functional and declarative intermediate representation (IR), and a Multi-Level Intermediate Representation (MLIR) code generator. The eCC++ front-end consists of a library of C++ classes and operators that can be used to define the guest language. Guest sources are compiled with any standard C++ compiler, and when run, the resulting executable generates an eCC++ IR representation of the program, which can be verified within the eCC++ framework. Finally, eCC++ allows for high-level and domain-specific optimizations before generating MLIR. In summary, eCC++ aims to act as a generic front-end that enables embedding guest languages into C++, and provides necessary compiler technology for program verification, targeting the existing capabilities in the MLIR infrastructure. The paper evaluates the eCC++ expressiveness and usability describing the process of embedding GraphIt, a high-performance graph language in C++.

Gonzalez Tallada, Marc↗
DOE OSTI2023

Survey of U.S. Research Reactor Auxiliary Facilities Used for Material Testing and Basic Neutron Science

A catalogue of and attributes of the Materials Irradiation and Testing facilities (MIF) and the Basic Neutron Science facilities (BSF) of auxiliary (AUX) facilities of research reactors are compiled. The survey of these facilities is drawn from the set of U.S. university and DOE research reactor facilities and should be considered as a reference point when comparing commonalities in international AUX facilities. The size and shielding capabilities of the facilities, the typical characterization equipment used, and the neutron flux and irradiation capabilities of the facilities are listed. Short descriptions of experimental activities and current practices are detailed for MIFs and BSFs. The attributes are important in consideration of the reconfiguration of the facilities for purposes other than stated mission, i.e., for proliferation of weapons-usable nuclear material (WUNM). An evaluation of the capacity for production rate of WUNM from neutron beams-on-targets or a sample that has been placed in an irradiation position for a period is provided. Further evaluation of reconfiguration of these MIF and BSF are recommended to refine the proliferation risks. Specifically, MIF hot cells and potential configurations, additional modeling of facility throughput and development of methods for determining levels of concern for MIF and BSF have all been identified as steps for refinement of determination of the risks associated with these facilities. Future work will focus on: 1. A model of projected throughput for different MIF and BSF configurations that will be used to provide a window of potential operational misuse for a facility and better understand the capability of production and the rate of processing for WUNM in these facilities. 2. A misuse study for various configurations of MIF hot cell layouts detailing capacity, specifications, shielding constraints for MIF operations, identification of a maximum shielding thickness, and reasonable need associated with each activity performed in a MIF. 3. Development of a rule of thumb rating system utilizing data gathered from previous reports to show levels of concern at a given power level, flux, and experiment set.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗