Search NASA⌕ Search

SEARCH · Search NASA

Results for “usability evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Usability Evaluation of the Innovation Portal and Integrated Capability Analysis Platform

The usability evaluation of the ICAP and IP tools was conducted as part of the United States (U.S.) Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) ION effort with two evaluators for the Idaho National Laboratory. Many nuclear operations in the United States are facing challenges of increased business, operations, and maintenance costs due to subsidized renewables and natural gas. The commercial NPP industry has looked to other industries to develop methods such as Integrated Operations (IO) to use available technologies to reduce operating costs. One concept for reducing costs that has been developed is the Integrated Operations for Nuclear (ION). A capability Stack model was developed as part of the ION effort to break down information related to People, Technology, Processes, and Governance (PTPG). The Innovation Portal (IP) and Integrated Capability Analysis Platform (ICAP) were created from the Capability Stack model as part of the ION effort to help utilities reduce O&M costs by identifying work reduction opportunities. Two heuristic evaluations and a cognitive walkthrough of the ICAP and IP tools were completed to identify usability issues to further develop the tools for use in the ION effort. Usability issues identified were rated by priority levels and across both tools. The priority levels identified were high (5), medium (11), and low (10). Recommendation were made to help reduce usability issues of the systems and help increase the user response rate of using the systems once they are implemented for utilities. Future work on designs for the IP tool and additional heuristic evaluations and external testing are being developed to continue refinement of the ICAP and IP tools for the ION effort.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2022

Operator Insights and Usability Evaluation of Machine Learning Assistance for Power Grid Contingency Analysis

Introducing machine learning (ML) assistance into any established process comes with adoption barriers, including entrenched procedures, technological and human readiness levels, human-machine trust, and work culture resistance to change. These barriers are even greater in critical operations such as operating a national or regional power grid, in which both regulatory frameworks and the importance of maintaining reliability levels causes additional resistance to the adoption of new computational support. Developers of future systems and job aides must consider not only technical aspects, but also whether new systems are usable by power system operators. This work presents the methodology and results of a study to evaluate the usability and readiness of a prototype recommender system for power grid contingency analysis. We explore operator cognitive load and evaluate operator performance when solving a collection of scenarios both with and without recommender assistance. We also examine operator trust in the system. We report insights gained on the readiness of the system using a collection of evaluation techniques.

Human-Machine Teaming, Power Systems, usability ev↗
DOE OSTI2017

User-Centered Evaluation of Visual Analytics

Visual analytics systems are becoming very popular. More domains now use interactive visualizations to analyze the ever-increasing amount and heterogeneity of data. More novel visualizations are being developed for more tasks and users. We need to ensure that these systems can be evaluated to determine that they are both useful and usable. A user-centered evaluation for visual analytics needs to be developed for these systems. While many of the typical human-computer interaction (HCI) evaluation methodologies can be applied as is, others will need modification. Additionally, new functionality in visual analytics systems needs new evaluation methodologies. There is a difference between usability evaluations and user-centered evaluations. Usability looks at the efficiency, effectiveness, and user satisfaction of users carrying out tasks with software applications. User-centered evaluation looks more specifically at the utility provided to the users by the software. This is reflected in the evaluations done and in the metrics used. In the visual analytics domain this is very challenging as users are most likely experts in a particular domain, the tasks they do are often not well defined, the software they use needs to support large amounts of different kinds of data, and often the tasks last for months. These difficulties are discussed more in the section on User-centered Evaluation. Our goal is to provide a discussion of user-centered evaluation practices for visual analytics, including existing practices that can be carried out and new methodologies and metrics that need to be developed and agreed upon by the visual analytics community. The material provided here should be of use for both researchers and practitioners in the field of visual analytics. Researchers and practitioners in HCI and interested in visual analytics will find this information useful as well as a discussion on changes that need to be made to current HCI practices to make them more suitable to visual analytics. A history of analysis and analysis techniques and problems is provided as well as an introduction to user-centered evaluation and various evaluation techniques for readers from different disciplines. The understanding of these techniques is imperative if we wish to support analysis in the visual analytics software we develop. Currently the evaluations that are conducted and published for visual analytics software are very informal and consist mainly of comments from users or potential users. Our goal is to help researchers in visual analytics to conduct more formal user-centered evaluations. While these are time-consuming and expensive to carryout, the outcomes of these studies will have a defining impact on the field of visual analytics and help point the direction for future features and visualizations to incorporate. While many researchers view work in user-centered evaluation as a less-than-exciting area to work, the opposite is true. First of all, the goal is user-centered evaluation is to help visual analytics software developers, researchers, and designers improve their solutions and discover creative ways to better accommodate their users. Working with the users is extremely rewarding as well. While we use the term “users” in almost all situations there are a wide variety of users that all need to be accommodated. Moreover, the domains that use visual analytics are varied and expanding. Just understanding the complexities of a number of these domains is exciting. Researchers are trying out different visualizations and interactions as well. And of course, the size and variety of data are expanding rapidly. User-centered evaluation in this context is rapidly changing. There are no standard processes and metrics and thus those of us working on user-centered evaluation must be creative in our work with both the users and with the researchers and developers.

intelligence analysis↗
DOE OSTI2024

Modeling the impact of extreme summer drought on conventional and renewable generation capacity: Methods and a case study on the Eastern U.S. power system

Across recent years, there has been a growing prevalence of extreme weather events throughout the United States, posing significant challenges to the reliable and resilient operation of power systems. Specifically, summer droughts threaten to severely reduce available generation capacity to meet regional electricity demand, potentially leading to power outages. This underscores the importance of accurate resource adequacy (RA) assessment to ensure the reliable operation of the nation’s energy infrastructure. Accurately evaluating the usable capacity of regional generation fleets is a challenging undertaking due to the intricate interactions between power systems and hydro-climatic systems. Here, this paper proposes a systematic and analytical framework to evaluate the impacts of extreme summer drought events on the available capacity of various generating technologies, incorporating both meteorological and hydrologic factors. The framework provides detailed plant-level capacity derating models for hydroelectric, thermoelectric, and renewable power plants, facilitating evaluations with high temporal and spatial resolution. The application of the proposed impact assessment framework to the 2025 generation fleet of the real-world power system within the PJM and SERC regions of the United States yields insightful results. By analyzing the daily usable capacity of 6,055 at-risk generators across the study region, it shows that the summer capacity deration is most significant for hydroelectric and once-through thermal power plants, followed by recirculating thermal power plants and combustion turbines. In the event of the recurrence of the 2007 southeastern summer drought event in the near future, the generation fleet could experience a substantial reduction in available capacity, estimated at approximately 8.5 GW, compared to typical summer conditions. The sensitivity analysis reveals that the usable capacity of the generation fleet would suffer an even more significant decrease under conditions of increasingly severe summer droughts. The proposed approach and the findings of this study provide valuable methodologies and insights, empowering stakeholders to bolster the resilience of power systems against the potentially devastating effects of future extreme drought events.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Reliability of Open Public Electric Vehicle Direct Current Fast Chargers

The aim was to systematically evaluate the usability of all public electric vehicles (EV) direct current fast chargers (DCFC) in the San Francisco region. To achieve a rapid transition to EVs, a highly reliable and easy to use charging infrastructure is critical to building confidence among consumers. The functionality and usability of all 182 open, public DCFC charging stations with CCS connectors (combined charging system) in the 9 counties of the Bay Area were tested (655 electric vehicle service equipment (EVSE) ports). An EVSE was classified as functional if it charged an EV for 2 minutes. Overall, 73.3% of the 655 EVSEs were functional. The causes of the nonfunctioning EVSEs (23.5%) were blank or unresponsive screens or error messages; payment system failures; charge initiation failures; network failures; or broken connectors. In addition, the cable was too short to reach the EV inlet for 3.2% of the EVSEs. A random sampling of 10% of the EVSEs, approximately 8 days after the first evaluation, found no overall change in functionality. The level of functionality found with field testing conflicts with the 95–98% uptime reported by the EV service providers (EVSPs) who operate the EV charging stations. There is a need for precise and verifiable definitions of uptime, downtime, and excluded time, as applied to public EV chargers. In conclusion, the level of failure of the existing public EV DCFC charge infrastructure highlights the importance of improving the system design and maintenance to improve adoption of EVs.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2017

Application-level regression testing framework using Jenkins

Monitoring and testing for regression of large-scale systems such as the NCSA's Blue Waters supercomputer are challenging tasks. In this paper, we describe the solution we came up with to perform those tasks. The goal was to find an automated solution for running user-level regression tests to evaluate system usability and performance. Jenkins, an automation server software, was chosen for its versatility, large user base, and multitude of plugins including collecting data and plotting test results over time. We also describe our Jenkins deployment to launch and monitor jobs on remote HPC system, perform authentication with one-time password, and integrate with our LDAP server for its authorization. We show some use cases and describe our best practices for successfully using Jenkins as a user-level system-wide regression testing and monitoring framework for large supercomputer systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Examining the Use of the Technology Acceptance Model for Adoption of Advanced Digital Technologies in Nuclear Power Plants

The United States nuclear industry is being economically challenged and needs wide-scale adoption of enabling technology like advanced automation to remain viable. To enable wide-scale adoption of advanced technology, addressing the human-technology integration challenges and factors that influence technology acceptance (i.e., ‘buy in’) is pertinent. This work examines the Technology Acceptance Model as a framework for characterizing the factors influencing technology adoption in the nuclear industry. An outcome of this work is to support the development of future tools such as a technology acceptance survey that can be used to assess technology acceptance throughout the lifespan of a modernization program to provide a systematic way of ensuring stakeholder and operations ‘buy in.’

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

HydrogenAR: Interactive Data-Driven Presentation of Dispenser Reliability: Preprint

When delivering presentations to a co-located audience, we typically use slides with text and 2D graphics to complement the spoken narrative. Though presentations have largely been explored on 2D media, augmented reality (AR) allows presentation designers to add data and augmentations to existing physical infrastructure on display. This could provide a more engaging experience to the audience and support comprehension. With HydrogenAR, we present a novel application that leverages the benefits of data-driven storytelling with those of AR to explain the unique challenges of hydrogen dis- penser reliability. Utilizing physical props, situated data, and virtual augmentations and animations, HydrogenAR serves as a unique presentation tool, particularly critical for stakeholders, tour groups, and VIPs. HydrogenAR is a product of multiple collaborative de- sign iterations with a local Hydrogen Fuel research team and is evaluated through interviews with team members and a user study with end-users to evaluate the usability and quality of the interactive AR experience. Through this work, we provide design considera- tions for AR data-driven presentations and discuss how AR could be used for innovative content delivery beyond traditional slide-based presentations.

augmented reality↗
DOE OSTI2018

Record High Hydrogen Storage Capacity in the Metal–Organic Framework Ni 2 ( m -dobdc) at Near-Ambient Temperatures

Hydrogen holds promise as a clean alternative automobile fuel, but its on-board storage presents significant challenges due to the low temperatures and/or high pressures required to achieve a sufficient energy density. The opportunity to significantly reduce the required pressure for high density H 2 storage persists for metal-organic frameworks due to their modular structures and large internal surface areas. The measurement of H 2 adsorption in such materials under conditions most relevant to on-board storage is crucial to understanding how these materials would perform in actual applications, although such data have to date been lacking. In the present work, the metal-organic frameworks M 2 (m-dobdc) (M = Co, Ni; m-dobdc 4- = 4,6-dioxido-1,3-benzenedicarboxylate) and the isomeric frameworks M 2 (dobdc) (M = Co, Ni; dobdc 4- = 1,4-dioxido-1,3-benzenedicarboxylate), which are known to have open metal cation sites that strongly interact with H 2 , were evaluated for their usable volumetric H 2 storage capacities over a range of near-ambient temperatures relevant to on-board storage. Based upon adsorption isotherm data, Ni 2 (m-dobdc) was found to be the top-performing physisorptive storage material with a usable volumetric capacity between 100 and 5 bar of 11.0 g/L at 25 °C and 23.0 g/L with a temperature swing between -75 and 25 °C. Additional neutron diffraction and infrared spectroscopy experiments performed with in situ dosing of D 2 or H 2 were used to probe the hydrogen storage properties of these materials under the relevant conditions. Finally, the results provide benchmark characteristics for comparison with future attempts to achieve improved adsorbents for mobile hydrogen storage applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2018

A low cost, re-usable electricity-free infant warmer: evaluation of safety, effectiveness and feasibility

Setting: Rural Rwandan hospitals, where thermoregulation is critical yet a challenge for pre-term, low-birth-weight (LBW) or sick newborns. Objective: To assess the safety, effectiveness, and feasibility of an inexpensive, reusable, non-electric warmer to complement kangaroo mother care (KMC). Methods: Prospective single-arm, non-randomized intervention study. Enrolled infants were hypothermic or at risk of hypothermia due to prematurity/LBW. Infants used the warmer in conjunction with KMC or as the sole source of external heat. Temperatures of the infant, warmer and air were measured for up to 6 h. Results: Overall, 33 patients used the warmer for 102 encounters: 43 hypothermic and 59 at risk of hypothermia. In 7/102 encounters (7%), the infant developed a temperature of >37.5°C (37.6°-38.2°C). For 43 hypothermic encounters and 59 at-risk encounters, hypothermia was corrected/prevented in respectively 41 (95%) and 59 (100%) instances. The warmer maintained goal temperature for the study duration in 85% of uses. Two/12 warmers broke down after <10 uses. In no instances was the warmer used incorrectly. Conclusion: Our results are promising for this prototype design, and warrant testing on a wider scale.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2017

Performance evaluation of an agent-based occupancy simulation model

Occupancy is an important factor driving building performance. Static and homogeneous occupant schedules, commonly used in building performance simulation, contribute to issues such as performance gaps between simulated and measured energy use in buildings. Stochastic occupancy models have been recently developed and applied to better represent spatial and temporal diversity of occupants in buildings. However, there is very limited evaluation of the usability and accuracy of these models. This study used measured occupancy data from a real office building to evaluate the performance of an agent-based occupancy simulation model: the Occupancy Simulator. The occupancy patterns of various occupant types were first derived from the measured occupant schedule data using statistical analysis. Then the performance of the simulation model was evaluated and verified based on (1) whether the distribution of observed occupancy behavior patterns follows the theoretical ones included in the Occupancy Simulator, and (2) whether the simulator can reproduce a variety of occupancy patterns accurately. Results demonstrated the feasibility of applying the Occupancy Simulator to simulate a range of occupancy presence and movement behaviors for regular types of occupants in office buildings, and to generate stochastic occupant schedules at the room and individual occupant levels for building performance simulation. For future work, model validation is recommended, which includes collecting and using detailed interval occupancy data of all spaces in an office building to validate the simulated occupant schedules from the Occupancy Simulator.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2017

Limitations of the clump-correlation theories of shear-induced turbulence suppression

The clump theory, primarily constructed by Dupree [Phys. Fluids 15, 334 (1972)] based on the moment approach and then generalized to the correlation theory [Y. Z. Zhang and S. M. Mahajan, Phys. Fluids B 5, 2000 (1993)], has long served as a basis for constructing theories of turbulence suppression by shear flow. In order to reveal the “intrinsic approximation” invoked in the clump-correlation theory, we examine a model based on two dimensional magnetized drift waves. After a rigorous derivation of the exact response function—a key to average the Green function of the system—we show that the Dupree, Zhang-Mahajan approach is recovered as the lowest order approximation in a small dimensionless parameter $Υ$ which is a triple product of the correlation time, wave number, and fluctuating drift velocity. The clump-correlation theory, thus, constitutes the Gaussian and lowest order non-Markovian process for a homogeneous stationary turbulence. Lastly we also provide, especially for the tokamak community, a readily usable formula to evaluate the effectiveness of shear-flow suppression; this formula pertains regardless of the specific model of correlation time.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Assessment of Electric Grid Transmission System Simulator for Human Factors Research

Over the past decades, various technologies have been developed for electric grid operations to support clean energy, meet rising electricity demands, and address infrastructure concerns. However, the human factors aspect is often overlooked during rapid integration. Questions persist about how these technologies impact human performance. Simulators play a critical role in supporting investigation of human factors design concepts and conducting comprehensive usability testing to evaluate human performance and assess human reliability. This paper aims to address human factors research simulator requirements and conduct a comparative study of six different simulators. A detailed evaluation reveals that the evaluated simulators lack the ability to customize user interfaces. Additionally, their user interface designs do not fulfill the basic human factors design principles, potentially leading to increased response variability and reduced statistical power when conducting controlled experimental research. In the future, it is essential to develop scripting tools to integrate customizable user interfaces and simulation models, ensuring meeting research requirements.

Li, Ruixuan↗
DOE OSTI2020

An Integrated Energy Systems Prototype Human-System Interface for a Steam Extraction Loop System to Support Joint Electricity-Hydrogen Flexible Operations

Due to increasing economic competition from renewables and combined-cycle natural gas plants, nuclear power plants are looking toward flexible operations to enhance their cost competitiveness. The Integrated Energy Systems project under the Light Water Reactor Sustainability program of the U.S. Department of Energy focuses on joint electricity-hydrogen flexible operations. Joint electricity-hydrogen flexible operations entail the nuclear power plant diverting thermal energy via main steam to a hydrogen production plant located nearby. The steam serves to enhance the efficiency of the hydrogen production. Furthermore, high temperature electrolysis requires a large amount of electricity, which the plant can also provide. The plant provides steam and electricity to the hydrogen plant throughout the day, but during peak demand hours the nuclear power plant returns to solely providing electricity to meet the high demand. Through this flexible concept of operations, the plant can optimize the thermal energy it produces without having to maneuver the power of the plant. This report documents the human factors process to design and develop a prototype human-system interface for the steam extraction loop that serves as the conduit between the nuclear power plant and the adjacent hydrogen plant. The design process followed the human factors guidelines set by NUREG-0711, Human Factors Engineering Program Review Model (O’Hara, Higgins, & Fleger, 2012), and expanded upon by the Guideline for Operational Nuclear Usability and Knowledge Elicitation (GONUKE; Boring, Ulrich, Joe, & Lew, 2015; Boring, Lew, & Ulrich, 2016). The design process entailed operator interviews to determine the concept of operations for the steam extraction loop, a review and adaptation of digital interface design concepts developed by the team in prior projects, an iterative design process, and reviews conducted by both operators and human factors experts. Several versions of the prototype human-system interface were developed. Operators were interviewed to determine what design features they found useful and would like to see in the interface. The design underwent a review by human factors experts against NUREG-0700, Human Interface Design Review Guidelines (U.S. Nuclear Regulatory Commission, 2019), to ensure compliance with the latest human factors standards for digital interfaces in nuclear applications. The design was then prototyped as a functional windows-based application integrated with the Generic Pressurized Water Reactor simulator modified to include the steam extraction loop. The simulation is supported by the Human Systems Simulation Laboratory at Idaho National Laboratory, which supports operator-in-the-loop testing. This is an ongoing project and the next phase of the project entails performing an operator-in-the-loop usability study to evaluate the interface and examine the proposed concept of operations to extraction steam from the nuclear power plant for delivery to the coupled hydrogen production plant.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2025

Advancing material modeling in hydrocodes using a concurrent finite-element and molecular dynamics multiscale framework

We present a multiscale simulation framework that couples the finite-element method with molecular dynamics. Bypassing traditional equations of state (EOS) by using in-line atomistic simulations, the method offers the advantage of incorporating detailed microscale physics not easily represented with coarse-grained models. Coupling consistency with the continuum code is ensured through the use of lifting and restriction operators, in line with heterogeneous multiscale methods. The concurrent continuum-atomistic framework is validated through comparison with experimental results and conventional EOS models, and demonstrated in a shock-driven hydrodynamic flow simulation under extreme conditions. We further evaluate the framework's usability by comparing it to state-of-the-art EOS models of deuterium. A computational performance study reveals that the atomistic EOS evaluation is a feasible alternative to conventional approaches, and demonstrates a weak scaling of 99% efficiency. These results highlight the framework's potential for large-scale multiscale modeling across a broad range of materials and conditions.

Computer science↗
DOE OSTI2024

Distributed Neural Representation for Reactive In Situ Visualization

Implicit neural representations (INRs) have emerged as a powerful tool for compressing large-scale volume data. This opens up new possibilities for in situ visualization. However, the efficient application of INRs to distributed data remains an underexplored area. Here, in this work, we develop a distributed volumetric neural representation and optimize it for in situ visualization. Our technique eliminates data exchanges between processes, achieving state-of-the-art compression speed, quality and ratios. Our technique also enables the implementation of an efficient strategy for caching large-scale simulation data in high temporal frequencies, further facilitating the use of reactive in situ visualization in a wider range of scientific problems. We integrate this system with the Ascent infrastructure and evaluate its performance and usability using real-world simulations.

Wu, Qi↗
DOE OSTI2017

Comparison between EMPIRE and TENDL

This report is a review of the proposed TENDL-2015 additions to the ENDF/B-VIII.0 library. We also discuss additions and modifications that are necessary so the new evaluations can be made usable. For that we performed calculations in the fast neutron range for the isotopes present in the TENDL- 2015 library as well as for other isotopes that either also satisfied the half-life criterion adopted by TENDL or filled the gaps between them in the nuclide chart. The quality of such calculations was assessed and compared to those of TENDL-2015, based on which recommendations were made. Finally, the files were validated so they can be incorporated into the ENDF/B-VIII.0 release.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗