SEARCH · Search NASA

Results for “Usability evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRS2018

Evaluation of Crew-Centric Onboard Mission Operations Planning and Execution Tool: Year 2

Currently, mission planning for the International Space Station (ISS) is largely affected by ground operators in mission control. The task of creating a week-long mission plan for ISS crew takes dozens of people multiple days to complete, and is often created far in advance of its execution. As such, re-planning or adapting to changing real-time constraints or emergent issues is similarly taxing. As we design for future mission operations concepts to other planets or areas with limited connectivity to Earth, more of these ground-based tasks will need to be handled autonomously by the crew onboard.There is a need for a highly usable (including low training time) tool that enables efficient self-scheduling and execution within a single package. The ISS Program has identified Playbook as a potential option. It already has high crew acceptance as a plan viewer from previous analogs and can now support a crew self-scheduling assessment on ISS or on another mission. The goals of this work, a collaboration between the Human Research Program and the ISS Program, are to inform the design of systems for more autonomous crew operations and provide a platform for research on crew autonomy for future deep space missions. Our second year of the research effort have included new insights on the crew self-scheduling sessions performed by the crew through use on the HERA (Human Exploration Research Analog) and NEEMO (NASA Extreme Environment Mission Operations) analogs. Use on the NEEMO analog involved two self-scheduling strategies where the crew planned and executed two days of EVAs (Extra-Vehicular Activities). On HERA year two represented the first HERA campaign where we were able to perform research tasks. This involved selected flexible activities that the crew could schedule, mock timelines where the crew completed more complex planning exercises, usability evaluation of the crew self-scheduling features, and more insights into the limit of plan complexity that the crew could effectively self-schedule. In parallel we have added in new features and functionality in the Playbook tool based off of our insights from crew self-scheduling in the NASA analogs. In particular this year we have added in the ability for the crew to add, edit, and remove their own activities in the Playbook tool, expanding the type of planning and re-planning possible in the tool and opening up the ability for more free form plan creation. The ability to group and manipulate groups of activities from the plan task list was also added, allowing crew members to add predefined sets of activities onto their mission timeline. In addition we also added a way for crew members to roll back changes in their plan, in order to allow an undo like capability. These features expand and complement the initial self-scheduling features added in year one with the goal of making crew autonomous planning more efficient. As part of this work we have also finished developing the first version of our Playbook Data Analysis Tool, a research tool built to interpret and analyze the unobtrusively collected data obtained during the NASA analog missions through Playbook. This data which includes user click interaction as well as plan change information, through the Playbook Data Analysis Tool, allows us to playback this information as if a video camera was mounted over the crewmember's tablet. While the primary purpose of this tool is to allow usability analysis of crew self-scheduling sessions used on the NASA analog, since the data collected is structured, the tool can automatically derive metrics that would be traditionally tedious to achieve without manual analysis of video playback. We will demonstrate and discuss the ability for future derived metrics to be added to the tool. In addition to the current data and results gathered in year two we will also discuss the preparation and goals of our International Space Station (ISS) onboard technology demonstration with Playbook. This technology demonstration will be preformed as part of the CAST payload starting in late 2016.

onboard planning↗

NASA NTRS2003

A Comprehension Based Analysis of Autoflight System Interfaces

This cooperative agreement supported Dr. Peter Polson's participation in two interrelated research programs. The first was the development of the Situation-Goal-Behavior (SGB) Model that is both a formal description of an avionics system's logic and behavior and a representation of a system that can be understood by avionics designers, pilots, and training developers. The second was the development of a usability inspection method based on an approximate model, RAFIV, of pilot interactions with the Flight Management System (FMS). The main purpose of this report is to integrate the two models and provide a context in order to better characterize the accomplishments of this research program. A major focus of both the previous and this Cooperative Agreement was the development of usability evaluation methods that can be effectively utilized during all phases of the design, development, and certification process of modern avionics systems. The current efforts to validate these methods have involved showing that they generate useful analyses of known operational and training problems with the current generation of avionics systems in modern commercial airliners. This report is organized into seven sections. Following the overview, the second section describes the Goal-Situation-Behavior model and its applications. The next section summarizes the foundations of the RAFIV model and describes the model in some detail. The contents of both these sections are derived from previous reports referenced in footnotes. The fourth section integrates these two models into a complete design evaluation and training development framework. The fifth section contains conclusions and possible future directions for research. References are in Section 6. Section 7 contains the titles and abstracts of the papers paper describing in more detail the results of this research program.

Palmer, Everett↗

NASA NTRS2003

A Cockpit Display Designed to Enable Limited Flight Deck Separation Responsibility

Cockpit displays need to be substantially improved to serve the goals of situational awareness, conflict detection, and path replanning, in Free Flight. This paper describes the design of such an advanced cockpit display, along with an initial simulation based usability evaluation. Flight crews were particularly enthusiastic about color coding for relative altitude, dynamically pulsing predictors, and the use of 3-D flight plans for alerting and situational awareness.

Johnson, Walter W.↗

NASA NTRS1997

Increased User Satisfaction Through an Improved Message System

With all of the enhancements in software methodology and testing, there is no guarantee that software can be delivered such that no user errors occur, How to handle these errors when they occur has become a major research topic within human-computer interaction (HCI). Users of the Multimission Spacecraft Analysis Subsystem(MSAS) at the Jet Propulsion Laboratory (JPL), a system of X and motif graphical user interfaces for analyzing spacecraft data, complained about the lack of information about the error cause and have suggested that recovery actions be included in the system error messages...The system was evaluated through usability surveys and was shown to be successful.

Human-computer interaction error handling direct m↗

NASA NTRS2016

Design Concept for a Minimal Volume Spacecraft Cabin to Serve as a Mars Ascent Vehicle Cabin and Other Alternative Pressurized Vehicle Cabins

The Evolvable Mars Campaign is developing concepts for human missions to the surface of Mars. These missions are round-trip expeditions, thereby requiring crew launch via a Mars Ascent Vehicle (MAV). A study to identify the smallest possible pressurized cabin for this mission has developed a conceptual vehicle referred to as the minimal MAV cabin. The origin of this concept will be discussed as well as its initial concept definition. This will lead to a description of possible configurations to integrate the minimal MAV cabin with ascent vehicle engines and propellant tanks. Limitations of this concept will be discussed, in particular those that argue against the use of the minimal MAV cabin to perform the MAV mission. However, several potential alternative uses for the cabin are identified. Finally, recommended forward work will be discussed, including current work in progress to develop a full scale mockup and conduct usability evaluations.

Howard, Robert L., Jr.↗

NASA NTRS1999

Ergonomic Evaluation of the Foot Restraint Equipment Device (FRED)

Within the scope of the Microgravity Workstation and Restraint Evaluation project, funded by the NASA Headquarters Life Sciences Division, evaluations were proposed to be conducted in ground, KC-135, and/or Shuttle environments to investigate the human factors engineering (HFE) issues concerning confined/unique workstations, including crew restraint requirements. As part of these evaluations, KC-135 flights were conducted to investigate user/ workstation/ restraint integration for microgravity use of the FRED with the RMS workstation. This evaluation was a pre-cursor to Detailed Supplementary Objective (DSO) - 904 on STS-88. On that mission, a small-statured astronaut will be using the FRED restraint while working at the Aft RMS workstation. The DSO will collect video for later posture analyses, as well as subjective data in the form of an electronic questionnaire. This report describes the current FRED KC-135 evaluations. The primary objectives were to evaluate the usability of the FRED and to verify the DSO in-flight setup. The restraint interface evaluation consisted of four basic areas of restraint use: 1) adjustability; 2) general usability and comfort; 3) usability at the RMS workstation; and 4) assembly and disassembly.

Whitmore, Mihriban↗

NASA NTRS2006

Dynamic Tunnel Usability Study: Format Recommendations for Synthetic Vision System Primary Flight Displays

A usability study evaluating dynamic tunnel concepts has been completed under the Aviation Safety and Security Program, Synthetic Vision Systems Project. The usability study was conducted in the Visual Imaging Simulator for Transport Aircraft Systems (VISTAS) III simulator in the form of questionnaires and pilot-in-the-loop simulation sessions. Twelve commercial pilots participated in the study to determine their preferences via paired comparisons and subjective rankings regarding the color, line thickness and sensitivity of the dynamic tunnel. The results of the study showed that color was not significant in pilot preference paired comparisons or in pilot rankings. Line thickness was significant for both pilot preference paired comparisons and in pilot rankings. The preferred line/halo thickness combination was a line width of 3 pixels and a halo of 4 pixels. Finally, pilots were asked their preference for the current dynamic tunnel compared to a less sensitive dynamic tunnel. The current dynamic tunnel constantly gives feedback to the pilot with regard to path error while the less sensitive tunnel only changes as the path error approaches the edges of the tunnel. The tunnel sensitivity comparison results were not statistically significant.

Arthur, Jarvis J., III↗

NASA NTRSDate not supplied

The Role of Trust and Usability in Enabling Spaceflight Crew Autonomy

Future long duration exploration missions will require an increased use of onboard automated systems as spaceflight crews venture further than before and have longer communications delays with ground support. The design of these systems must support appropriate crew trust and have sufficient usability to enable spaceflight crew autonomy or risk being misused while crews wait to communicate with ground support. We evaluated trust & usability in our self-scheduling tool, Playbook, for crew mission timelines. Data was collected in a controlled lab experiment with 31 participants. Participants in the study conducted two tasks: scheduling, where participants were responsible for scheduling a majority of a day's operational tasks, and rescheduling, where participants were provided a schedule and asked to reschedule higher priority activities. We found a significant correlation between system trust and usability, irrespective of self-scheduling tasks. We conclude that system usability may play a bigger role in how trust is learned while conducting novel crew autonomy tasks such as self-scheduling. Future research should investigate the role of usability to encourage appropriate trust in onboard automated crew systems and enable crew autonomy.

crew autonomy↗

NASA NTRS1982

Ohio's Abandoned Mine Lands Reclamation Program: a Study of Data Collection and Evaluation Techniques

The planning process for a statewide reclamation plan of Ohio abandoned minelands in response to the Federal Surface Mining Control and Reclamation Act of 1977 included: (1) the development of a screening and ranking methodology; (2) the establishment of a statewide review of major watersheds affected by mining; (3) the development of an immediate action process; and (4) a prototypical study of a priority watershed demonstrating the data collection, analysis, display and evaluation to be used for the remaining state watersheds. Historical methods for satisfying map information analysis and evaluation, as well as current methodologies being used were discussed. Various computer mapping and analysis programs were examined for their usability in evaluating the priority reclamation sites. Hand methods were chosen over automated procedures; intuitive evaluation was the primary reason.

Sperry, S. L.↗

DOE OSTI2024

Modeling the impact of extreme summer drought on conventional and renewable generation capacity: Methods and a case study on the Eastern U.S. power system

Across recent years, there has been a growing prevalence of extreme weather events throughout the United States, posing significant challenges to the reliable and resilient operation of power systems. Specifically, summer droughts threaten to severely reduce available generation capacity to meet regional electricity demand, potentially leading to power outages. This underscores the importance of accurate resource adequacy (RA) assessment to ensure the reliable operation of the nation’s energy infrastructure. Accurately evaluating the usable capacity of regional generation fleets is a challenging undertaking due to the intricate interactions between power systems and hydro-climatic systems. Here, this paper proposes a systematic and analytical framework to evaluate the impacts of extreme summer drought events on the available capacity of various generating technologies, incorporating both meteorological and hydrologic factors. The framework provides detailed plant-level capacity derating models for hydroelectric, thermoelectric, and renewable power plants, facilitating evaluations with high temporal and spatial resolution. The application of the proposed impact assessment framework to the 2025 generation fleet of the real-world power system within the PJM and SERC regions of the United States yields insightful results. By analyzing the daily usable capacity of 6,055 at-risk generators across the study region, it shows that the summer capacity deration is most significant for hydroelectric and once-through thermal power plants, followed by recirculating thermal power plants and combustion turbines. In the event of the recurrence of the 2007 southeastern summer drought event in the near future, the generation fleet could experience a substantial reduction in available capacity, estimated at approximately 8.5 GW, compared to typical summer conditions. The sensitivity analysis reveals that the usable capacity of the generation fleet would suffer an even more significant decrease under conditions of increasingly severe summer droughts. The proposed approach and the findings of this study provide valuable methodologies and insights, empowering stakeholders to bolster the resilience of power systems against the potentially devastating effects of future extreme drought events.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS2014

Usability of EFBs for Viewing NOTAMs and AIS/MET Data Link Messages

Electronic Flight Bags (EFB) are increasingly integral to flight deck information management. A piloted simulation study was conducted at NASA Langley Research Center, one aspect of which was to evaluate the usability and acceptability of EFBs for viewing and managing Notices to Airmen (NOTAMs) and data linked aeronautical information services (AIS) and meteorological information (MET). The study simulated approaches and landings at Memphis International Airport (KMEM) using various flight scenarios and weather conditions. Ten two-pilot commercial airline crews participated, utilizing the Cockpit Motion Facility's Research Flight Deck (CMF/RFD) simulator. Each crew completed approximately two dozen flights over a two day period. Two EFBs were installed, one for each pilot. Study data were collected in the form of questionnaire/interview responses, audio/video recordings, oculometer recordings, and aircraft/system state data. Preliminary usability results are reported primarily based on pilot interviews and responses to questions focused on ease of learning, ease of use, usefulness, satisfaction, and acceptability. Analysis of the data from the other objective measures (e.g., oculometer) is ongoing and will be reported in a future publication. This paper covers how the EFB functionality was set up for the study; the NOTAM, AIS/MET data link, and weather messages that were presented; questionnaire results; selected pilot observations; and conclusions.

Evans, Emory T.↗

NASA NTRS2010

Usability Testing and Analysis Facility (UTAF)

This slide presentation reviews the work of the Usability Testing and Analysis Facility (UTAF) at NASA Johnson Space Center. It is one of the Space Human Factors Laboratories in the Habitability and Human Factors Branch (SF3) at NASA Johnson Space Center The primary focus pf the UTAF is to perform Human factors evaluation and usability testing of crew / vehicle interfaces. The presentation reviews the UTAF expertise and capabilities, the processes and methodologies, and the equipment available. It also reviews the programs that it has supported detailing the human engineering activities in support of the design of the Orion space craft, testing of the EVA integrated spacesuit, and work done for the design of the lunar projects of the Constellation Program: Altair, Lunar Electric Rover, and Outposts

Wong, Douglas T.↗

NASA NTRSDate not supplied

Developing and Validating Measurement Scales During Pandemic Conditions: A Case Study with the Scale for Habitat Usability

At NASA, habitat evaluations often employ subjective measures. Some measures are frequently used, well established tools, whereas others are homegrown measures tailored to specific projects. The variety of measures used makes evaluation comparisons across projects difficult. Additionally, some of these measures are burdensome, may be too specialized, or may require an expert to use and interpret, limiting their utility. Taken together, these drawbacks suggest the need for a new measurement tool. To that purpose, a team at NASA worked on developing a new scale for measuring habitat usability, the Scale for Habitat Usability (SHU). The SHU is intended to be a quick, multi-faceted measure for evaluating habitat usability across the development lifecycle. However, like many research projects, the development of the SHU faced setbacks due to the COVID-19 pandemic. Pandemic prevention protocols precluded in-person data collection, forcing the team to take some non-traditional approaches to scale development. This paper reports the steps the team took to complete the project.

Ian Robertson↗

NASA NTRS2004

NASA Johnson Space Center Usability Testing and Analysis Facility (UTAF) Overview

The Usability Testing and Analysis Facility (UTAF) is part of the Space Human Factors Laboratory at the NASA Johnson Space Center in Houston, Texas. The facility provides support to the Office of Biological and Physical Research, the Space Shuttle Program, the International Space Station Program, and other NASA organizations. In addition, there are ongoing collaborative research efforts with external businesses and universities. The UTAF provides human factors analysis, evaluation, and usability testing of crew interfaces for space applications. This includes computer displays and controls, workstation systems, and work environments. The UTAF has a unique mix of capabilities, with a staff experienced in both cognitive human factors and ergonomics. The current areas of focus are: human factors applications in emergency medical care and informatics; control and display technologies for electronic procedures and instructions; voice recognition in noisy environments; crew restraint design for unique microgravity workstations; and refinement of human factors processes. This presentation will provide an overview of ongoing activities, and will address how the projects will evolve to meet new space initiatives.

Whitmore, M.↗

NASA NTRS2004

NASA Johnson Space Center Usability Testing and Analysis Facility (WAF) Overview

Whitmore, M.↗

NASA NTRS2005

NASA Johnson Space Center Usability Testing and Analysis facility (UTAF) Overview

The Usability Testing and Analysis Facility (UTAF) is part of the Space Human Factors Laboratory at the NASA Johnson Space Center in Houston, Texas. The facility performs research for NASA's HumanSystems Integration Program, under the HumanSystems Research and Technology Division. Specifically, the UTAF provides human factors support for space vehicles, including the International Space Station, the Space Shuttle, and the forthcoming Crew Exploration Vehicle. In addition, there are ongoing collaborative research efforts with external corporations and universities. The UTAF provides human factors analysis, evaluation, and usability testing of crew interfaces for space applications. This includes computer displays and controls, workstation systems, and work environments. The UTAF has a unique mix of capabilities, with a staff experienced in both cognitive human factors and ergonomics. The current areas of focus are: human factors applications in emergency medical care and informatics; control and display technologies for electronic procedures and instructions; voice recognition in noisy environments; crew restraint design for unique microgravity workstations; and refinement of human factors processes and requirements. This presentation will provide an overview of ongoing activities, and will address how the UTAF projects will evolve to meet new space initiatives.

Whitmore, Mihriban↗

NASA NTRSDate not supplied

Rapid Assessment of a Lunar Surface Laboratory Module

Johnson Space Center is piloting an innovation team known as the Forge to perform rapid turnaround, concurrent engineering studies. The Forge is intended to be able to pull together a team of strategically selected domain experts to innovatively solve a specified problem within a short period of time. The Forge’s inaugural project being the development of a Lunar Surface Laboratory Module concept. The lab module is intended as an option for the Artemis program’s Lunar Architecture Team. The lab module may allow for increased science productivity and return on investment than a surface architecture with only the habitat. This paper will discuss the configuration of the Lunar Laboratory module, including the pressure vessel, its placement relative to the Surface Habitat and Pressurized Rover, and internal architecture. In order to determine the suitability of such a design, a human factors evaluation is conducted. Historically, a human factors evaluation of a habitable volume could require a month or longer, including time needed to prepare a test plan, conduct the evaluation, compile the data, and produce a report. The entire Forge effort will only last for five weeks, forcing the creation of a rapid assessment process. This rapid assessment includes three evaluation studies, all conducted and processed in the space of a single day, to determine the acceptability of the resulting design. The first evaluation is a card sort evaluation of the laboratory instruments. A brief discussion of the card sort evaluation is provided. This card sort is focused on the relative positioning of the laboratory instruments, determining whether a given instrument should be adjacent to, or separated from, another. It will use human factors personnel and both life and physical scientists as subject matter experts. The second evaluation is a science productivity tabletop evaluation. This evaluation focuses on the selection of instruments and the number of instruments, attempting to assess the amount of work the crew will be able to use the facility to do. It uses life and physical science personnel as subject matter experts. The third evaluation is a CAD or Virtual Reality human factors walkthrough evaluation. This evaluation focuses on the layout of the lab module and its connectivity to the basecamp’s pressurized elements. If possible, astronauts will be used as subject matter experts. Each of these evaluations will focus primarily on objective data with limited subjective comments, reducing the time required for processing responses. The results of each evaluation and their implications will be discussed. Based on the evaluation data, an overall acceptability of the Lunar Surface Laboratory Module produced by the Forge study can be determined. Lessons learned and recommendations for future use of the human factors rapid assessment process in early design trades will also be provided.

Lunar Laboratory↗

DOE OSTI2023

Reliability of Open Public Electric Vehicle Direct Current Fast Chargers

The aim was to systematically evaluate the usability of all public electric vehicles (EV) direct current fast chargers (DCFC) in the San Francisco region. To achieve a rapid transition to EVs, a highly reliable and easy to use charging infrastructure is critical to building confidence among consumers. The functionality and usability of all 182 open, public DCFC charging stations with CCS connectors (combined charging system) in the 9 counties of the Bay Area were tested (655 electric vehicle service equipment (EVSE) ports). An EVSE was classified as functional if it charged an EV for 2 minutes. Overall, 73.3% of the 655 EVSEs were functional. The causes of the nonfunctioning EVSEs (23.5%) were blank or unresponsive screens or error messages; payment system failures; charge initiation failures; network failures; or broken connectors. In addition, the cable was too short to reach the EV inlet for 3.2% of the EVSEs. A random sampling of 10% of the EVSEs, approximately 8 days after the first evaluation, found no overall change in functionality. The level of functionality found with field testing conflicts with the 95–98% uptime reported by the EV service providers (EVSPs) who operate the EV charging stations. There is a need for precise and verifiable definitions of uptime, downtime, and excluded time, as applied to public EV chargers. In conclusion, the level of failure of the existing public EV DCFC charge infrastructure highlights the importance of improving the system design and maintenance to improve adoption of EVs.

33 ADVANCED PROPULSION SYSTEMS↗