Results for “usability evaluation” — Page 8

NASA NTRS1984

Design criteria for a PC-based common user interface to remote information systems

A set of design criteria are presented which will allow the implementation of an interface to multiple remote information systems on a microcomputer. The focus of the design description is on providing the user with the functionality required to retrieve, store and manipulate data residing in remote information systems through the utilization of a standardized interface system. The intent is to spare the user from learning the details of retrieval from specific systems while retaining the full capabilities of each system. The system design includes multi-level capabilities to enhance usability by a wide range of users and utilizes microcomputer graphics capabilities where applicable. A data collection subsystem for evaluation purposes is also described.

Dominick, Wayne D.↗

NASA NTRSDate not supplied

QuARC: Development of a Service to Enable FAIR-er Metadata

The ARC Project: The ARC Team located at NASA’s Marshall Space Flight Center conducts quality assessments of metadata records that catalog NASA’s collection of over 9,000 Earth observation data products, stored in a centralized database called the Common Metadata Repository (CMR). The ARC Team has developed a metadata quality assessment framework to evaluate metadata completeness, correctness, and consistency with the goal of making NASA’s data products more discoverable, accessible, and usable. ARC = Analysis and Review of the CMR

Earth Science Informatics↗

NASA NTRS2005

Apex Reference Manual 3.0 Beta

Apex is a toolkit for constructing software that behaves intelligently and responsively in demanding task environments. Reflecting its origin at NASA where Apex continues to be developed, current applications include: a) Providing autonomous mission management and tactical control capabilities for unmanned aerial vehicles including an autonomous surveillance helicopter and a simulation prototype of an unmanned fixed-wing aircraft to be used for wildfire mapping; b) Simulating human air traffic controllers, pilots and astronauts to help predict how people might respond to changes in equipment or procedures; and c) Predicting the precise duration and sequence of routine human behaviors based on a human-computer interaction engineering technique called CPM-GOMS. Among Apex s components are a set of implemented reasoning services, such as those for reactive planning and temporal pattern recognition; a software architecture that embeds and integrates these services and allows additional reasoning elements to be added as extensions; a formal language for specifying agent knowledge; a simulation environment to facilitate prototyping and analysis; and Sherpa, a set of tools for visualizing autonomy logic and runtime behavior. In combination, these are meant to provide a flexible and usable framework for creating, testing, and deploying intelligent agent software. Overall, our goal in developing Apex is to lower economic barriers to developing intelligent software agents. New ideas about how to extend or modify the system are evaluated in terms of their impact in reducing the time, expertise, and inventiveness required to build and maintain applications. For example, potential enhancements to the AI reasoning capabilities in the system are reviewed not only for usefulness and distinctiveness, but also for their impact on the readability and general usability of Apex s behavior representation language (PDL) and on the transparency of resulting behavior. A second central part of our approach is to iteratively refine Apex based on lessons learned from as diverse a set of applications as possible. Many applications have been developed by users outside the core development team including engineers, researchers, and students. Usability is thus a central concern for every aspect of Apex visible to a user, including PDL, Sherpa, the Apex installation process, APIs, and user documentation. Apex users vary in their areas of expertise and in their familiarity with autonomy technology. Focusing on usability, a development philosophy summarized by the project motto "Usable Autonomy," has been important part of enabling diverse users to employ Apex successfully and to provide feedback needed to guide iterative, user-centered refinement.

Freed, Michael A.↗

NASA NTRS1990

Continuation of research into software for space operations support, volume 1

A prototype workstation executive called the Hardware Independent Software Development Environment (HISDE) was developed. Software technologies relevant to workstation executives were researched and evaluated and HISDE was used as a test bed for prototyping efforts. New X Windows software concepts and technology were introduced into workstation executives and related applications. The four research efforts performed included: (1) Research into the usability and efficiency of Motif (an X Windows based graphic user interface) which consisted of converting the existing Athena widget based HISDE user interface to Motif demonstrating the usability of Motif and providing insight into the level of effort required to translate an application from widget to another; (2) Prototype a real time data display widget which consisted of research methods for and prototyping the selected method of displaying textual values in an efficient manner; (3) X Windows performance evaluation which consisted of a series of performance measurements which demonstrated the ability of low level X Windows to display textural information; (4) Convert the Display Manager to X Window/Motif which is the application used by NASA for data display during operational mode.

Collier, Mark D.↗

NASA NTRS2023

A Summary of the Development and Validation of the Scale for Habitat Usability (SHU)

Usability is an important concept to consider when designing space habitats/vehicles. In addition to objective outcomes (success rate, net habitable volume etc.), collecting users’ subjective judgments via comments and questionnaires can provide valuable data for evaluating conceptual designs, identifying potential layout issues, improving concept of operations, substantiating design changes, or selecting between competing mockups. Historically at NASA, there has been no “gold standard” questionnaire for capturing users’ subjective viewpoint regarding habitat design. To address this gap a Human Research Program (HRP) study was conducted to create a new measurement tool: the Scale for Habitat Usability (SHU).

Usability↗

NASA NTRSDate not supplied

Improving GES Disc Data Search and Discovery Through AI Metadata Augmentation

NASA’s Goddard Earth Science (GES) Data and Information Services Center (DISC) is one of twelve data centers in NASA's Science Mission Directorate (SMD), providing vital earth science data to a diverse user base. To enhance the discoverability of this data, GES DISC employs a keyword search system, which leverages scientific keywords embedded in dataset metadata. However, the evolving nature of scientific applications of our data necessitates regular review and augmentation of these keywords. To address this, we developed a service to automatically predict missing science keywords in the metadata. This service constructs a knowledge graph from the latest GES DISC metadata within NASA’s Common Metadata Repository (CMR). Using an open-source library, we trained a machine learning model to predict absent science keywords in the metadata. Our preliminary results indicate that the model has high levels of accuracy at predicting science keywords in the dataset metadata when exposed to data not included in its training. These predicted keywords were then evaluated by GES DISC data curation scientists and compared against other AI tools for metadata augmentation. We aim to enhance the overall usability and accessibility of NASA’s earth science data by implementing this tool in our data curation processes.

Kendall Gilbert↗

NASA NTRSDate not supplied

Developing and Testing Two Interfaces for Supplemental Data Service Provider (SDSP) Tools to Support UAS Traffic Management (UTM)

Researchers conducted a usability study using two graphical user interfaces (GUIs) to explore how individuals interpret and interact with different preflight information displays, and to inform the development of Uncrewed Aircraft System (UAS) preflight planning predictive support tools to assess and mitigate flight hazards and risks. A series of preflight risk-assessment tasks were developed to evaluate participant performance using the Supplemental Data Service Provider-Consolidated Dashboard (SDSP-CD) and the Human Automation Team Interface System (HATIS) GUIs. Participants were trained to use both interfaces and their performance was evaluated. These evaluations focused on participants’ preflight planning activities. Objective data on performance tasks across different scenarios involving multi-UASs, as well as self-reports of interactions and subjective experiences using the GUIs were collected. Scores on the system usability scale (SUS) and on a simple task set were examined, as well as user feedback on open-ended questions, to inform development and identify potential improvements to the interfaces.

sUAAV interfaces↗

NASA NTRSDate not supplied

Developing and Testing Two Interfaces for Supplemental Data Service Provider (SDSP) Tools to Support UAS Traffic Management (UTM)

Researchers conducted a usability study using two graphical user interfaces (GUIs) to explore how individuals interpret and interact with different preflight information displays, and to inform the development of Uncrewed Aircraft System (UAS) preflight planning predictive support tools to assess and mitigate flight hazards and risks. A series of preflight risk-assessment tasks were developed to evaluate participant performance using the Supplemental Data Service Provider-Consolidated Dashboard (SDSP-CD) and the Human Automation Team Interface System (HATIS) GUIs. Participants were trained to use both interfaces and their performance was evaluated. These evaluations focused on participants’ preflight planning activities. Objective data on performance tasks across different scenarios involving multi-UASs, as well as self-reports of interactions and subjective experiences using the GUIs were collected. Scores on the system usability scale (SUS) and on a simple task set were examined, as well as user feedback on open-ended questions, to inform development and identify potential improvements to the interfaces.

sUAAV interfaces↗

NASA NTRSDate not supplied

Examining Performance and Usability of Situation Awareness Communication Tool for NASA's High Density Vertiplex (HDV) Team

Purpose: As cities get more crowded, the roadway infrastructure cannot keep up with the travel demands. Aviation can be a solution. Organizations supporting NASA’s Urban Air Mobility (UAM) concept are conducting studies on feasible concepts of operations for the new air traffic management system required to implement UAM. NASA’s High Density Vertiplex (HDV) team conducts studies using both live flights and simulations. Significance: The goal of this study is to evaluate the effectiveness of a communication tool developed for this team to enhance team situation awareness. Specifically, the study aims to determine if the communication tool is effective in allowing NASA-Langley and NASA-Ames team members playing different roles to understand what others are doing so that completed steps can be communicated to the whole team. Methods: This study asks participants to conduct four distinct scenarios using an Adobe XD prototype. Participants included five members of the target audience, who each received up to 15 minutes of training on how to use the Adobe XD prototype. The participants were NASA-Ames or NASA-Langley researchers, developers or tech leads who had 0 to 15 years of experience. Their ages ranged from 20 to 40 years old, and they were of both genders. After training, each participant was asked to do 4 tasks of varying difficulty, using the prototype testbed. After each task they reported on task difficulty and after all tasks were completed they were asked to fill out the System Usability Survey (SUS). Performance will be assessed through task completion time and accuracy rating. Results: User perception was evaluated by the SUS which showed a mean score over the six participants of 84. All the participants were above the acceptable range which was 70 and nobody was in the marginal usability zone between 50 and 70. In addition, most of the participants did well in terms of time on task, success rate and task difficulty on all the tasks except for Task 3. Participants had two minutes to do the task correctly to succeed at the task, otherwise it was marked as a failure. The geomean for Task 1 for completion time was 22.20; Task 2 was 58.28; Task 3 was 84.68 and Task 4 was 19.61. In Task 3, three out of six people failed this task but all five scored this with an average score 4.2 out of 5 for self-rating on task difficulty.

human factors↗

NASA NTRS2000

Evaluation of DVD-R for Archival Applications

For more than a decade, CD-ROM and CD-R have provided an unprecedented level of reliability, low cost and cross-platform compatibility to support federal data archiving and distribution efforts. However, it should be remembered that years of effort were required to achieve the standardization that has supported the growth of the CD industry. Incompatibilities in the interpretation of the ISO-9660 standard on different operating systems had to be dealt with, and the imprecise specifications in the Orange Book Part n and Part Hi led to incompatibilities between CD-R media and CD-R recorders. Some of these issues were presented by the authors at Optical Data Storage '95. The major current problem with the use of CD technology is the growing volume of digital data that needs to be stored. CD-ROM collections of hundreds of volumes and CD-R collections of several thousand volumes are becoming almost too cumbersome to be useful. The emergence of Digital Video Disks Recorder (DVD-R) technology promises to reduce the number of discs required for archive applications by a factor of seven while providing improved reliability. It is important to identify problem areas for DVD-R media and provide guidelines to manufacturers, file system developers and users in order to provide reliable data storage and interchange. The Data Distribution Laboratory (DDL) at NASA's Jet Propulsion Laboratory began its evaluation of DVD-R technology in early 1998. The initial plan was to obtain a DVD-Recorder for preliminary testing, deploy reader hardware to user sites for compatibility testing, evaluate the quality and longevity of DVD-R media and develop proof-of-concept archive collections to test the reliability and usability of DVD-R media and jukebox hardware.

Martin, Michael D.↗

NASA NTRS2007

Anisotropic Solution Adaptive Unstructured Grid Generation Using AFLR

An existing volume grid generation procedure, AFLR3, was successfully modified to generate anisotropic tetrahedral elements using a directional metric transformation defined at source nodes. The procedure can be coupled with a solver and an error estimator as part of an overall anisotropic solution adaptation methodology. It is suitable for use with an error estimator based on an adjoint, optimization, sensitivity derivative, or related approach. This offers many advantages, including more efficient point placement along with robust and efficient error estimation. It also serves as a framework for true grid optimization wherein error estimation and computational resources can be used as cost functions to determine the optimal point distribution. Within AFLR3 the metric transformation is implemented using a set of transformation vectors and associated aspect ratios. The modified overall procedure is presented along with details of the anisotropic transformation implementation. Multiple two-and three-dimensional examples are also presented that demonstrate the capability of the modified AFLR procedure to generate anisotropic elements using a set of source nodes with anisotropic transformation metrics. The example cases presented use moderate levels of anisotropy and result in usable element quality. Future testing with various flow solvers and methods for obtaining transformation metric information is needed to determine practical limits and evaluate the efficacy of the overall approach.

Marcum, David L.↗

NASA NTRS2020

Honeywell Enhancing Airplane State Awareness (EASA) Project: Final Report on Refinement and Evaluation of Candidate Solutions for Airplane System State Awareness

The loss of pilot airplane state awareness (ASA) has been implicated as a factor in several aviation accidents identified by the Commercial Aviation Safety Team (CAST). These accidents were investigated to identify precursors to the loss of ASA and develop technologies to address the loss of ASA. Based on a gap analysis, two technologies were prototyped and assessed with a formative pilot-in-the-loop evaluation in NASA Langley’s full-motion Research Flight Deck. The technologies address: 1) data source anomaly detection in real-time, and 2) intelligent monitoring aids to provide nominal and predictive awareness of situations to be monitored and a mission timeline to visualize events of interest. The evaluation results indicated favorable impressions of both technologies for mitigating the loss of ASA in terms of operational utility, workload, acceptability, complexity, and usability. The team concludes that there is a feasible retrofit solution for improving ASA that would minimize certification risk, integration costs, and training impact.

Whitlow, Stephen D.↗

NASA NTRSDate not supplied

Flight Deck Design of a Hybrid Turbine/Electric Passenger Aircraft

NASA is exploring the development of a 180-passenger subsonic single engine aft turbine aircraft, The aft turbine provides electric power in a hybrid design to wing mounted electric engines, creating a highly efficient, high-bypass-ratio fan equivalent. The SUbsonic Single Aft eNgine (SUSAN) aircraft is being developed as a sustainable subsonic regional aircraft that seeks to reduce emission levels by 50% in the next few decades. Pilot-in-the-loop studies were conducted at the NASA Langley Research Center in Hampton, Virginia, to explore the flight deck design for the hybrid electric aircraft. Following modern trends in commercial aircraft flight decks with full time augmented controls and a quiet and dark philosophy, single throttle and simplified engine displays were developed for the SUSAN aircraft. The aircraft includes a single aft mounted turbine engine and 16 wing mounted electric fans. The final design was developed from feedback received during an earlier pilot-in-the-loop study where one, two, and three throttles were tested in standard airline operations, including various failures of the turbine and electric engines. Current flight deck designs normally provide control inceptors for each propulsion engine and an engine display for all primary aircraft engine parameters. With full time augmentation expected, a single throttle control with autothrottle always engaged, even during failures, is desired. Augmentation of flight controls using distributed thrust also requires full time control of the electric engines using automation. Additionally, electric engine thrust is augmented during climb based on battery state of charge. Thrust augmentation changes faster than human reaction time and therefore requires full-time automation. A pilot-in-the-loop study was conducted at the NASA Langley Research Center in Hampton, Virginia, to test the final design of the single throttle with simplified engine displays. Fourteen airline pilots evaluated the single throttle and engine display concept. Electric engine failures included one, four symmetric, and eight non-symmetric electric engine failures. The turbine engine was evaluated for complete and partial failure during critical phases of flight to include takeoff as well as enroute. Unexpected go-arounds increase workload and require significant throttle manipulation. Go-arounds were included to ensure the single throttle was usable for all phases of flight. Failures during takeoff required a return to the departure field and failures enroute required a diversion except for one and four electric engine failures as these failures did not affect aircraft flyability or range. There are currently no Part 25 aircraft certified with hybrid systems or electric engines with batteries as emergency propulsion. For turbine engine failures in the SUSAN aircraft design, range is limited to 30 minutes at full power. Battery state of charge and battery health displays were developed and tested for usability and to determine how well they supported pilot decisions for alternate airports during emergency diversions. Novel displays using shape and color were developed to provide immediate feedback when state of charge became critical. This paper details the pilot study including pilot feedback supporting the potential for increased automation and a single throttle control. Detailed recommendations are provided for a novel single throttle control and additional pilot controls to support selection of engines during start, shutdown, and engine troubleshooting procedures. This design deviates significantly from current practice of providing throttles for each propulsion engine. Engine display recommendations are provided based on pilot feedback during a guided post-evaluation interview. Battery state of charge and battery health display recommendations were collected from all airline crews. The simplified engine displays design was rated excellent as measured with a usability scale. Quantitative metrics include airspeed tracking, time to complete checklists, time to make diversion decisions and the quality of the diversion decision. Recommendations for future studies are documented with supporting research and current observations about upcoming flight deck certifications.

autothrottle↗

NASA NTRSDate not supplied

Flight Deck Design of a Hybrid Turbine/Electric Passenger Aircraft

NASA is exploring the development of a 180-passenger subsonic single engine aft turbine aircraft, The aft turbine provides electric power in a hybrid design to wing mounted electric engines, creating a highly efficient, high-bypass-ratio fan equivalent. The SUbsonic Single Aft eNgine (SUSAN) aircraft is being developed as a sustainable subsonic regional aircraft that seeks to reduce emission levels by 50% in the next few decades. Pilot-in-the-loop studies were conducted at the NASA Langley Research Center in Hampton, Virginia, to explore the flight deck design for the hybrid electric aircraft. Following modern trends in commercial aircraft flight decks with full time augmented controls and a quiet and dark philosophy, single throttle and simplified engine displays were developed for the SUSAN aircraft. The aircraft includes a single aft mounted turbine engine and 16 wing mounted electric fans. The final design was developed from feedback received during an earlier pilot-in-the-loop study where one, two, and three throttles were tested in standard airline operations, including various failures of the turbine and electric engines. Current flight deck designs normally provide control inceptors for each propulsion engine and an engine display for all primary aircraft engine parameters. With full time augmentation expected, a single throttle control with autothrottle always engaged, even during failures, is desired. Augmentation of flight controls using distributed thrust also requires full time control of the electric engines using automation. Additionally, electric engine thrust is augmented during climb based on battery state of charge. Thrust augmentation changes faster than human reaction time and therefore requires full-time automation. A pilot-in-the-loop study was conducted at the NASA Langley Research Center in Hampton, Virginia, to test the final design of the single throttle with simplified engine displays. Fourteen airline pilots evaluated the single throttle and engine display concept. Electric engine failures included one, four symmetric, and eight non-symmetric electric engine failures. The turbine engine was evaluated for complete and partial failure during critical phases of flight to include takeoff as well as enroute. Unexpected go-arounds increase workload and require significant throttle manipulation. Go-arounds were included to ensure the single throttle was usable for all phases of flight. Failures during takeoff required a return to the departure field and failures enroute required a diversion except for one and four electric engine failures as these failures did not affect aircraft flyability or range. There are currently no Part 25 aircraft certified with hybrid systems or electric engines with batteries as emergency propulsion. For turbine engine failures in the SUSAN aircraft design, range is limited to 30 minutes at full power. Battery state of charge and battery health displays were developed and tested for usability and to determine how well they supported pilot decisions for alternate airports during emergency diversions. Novel displays using shape and color were developed to provide immediate feedback when state of charge became critical. This paper details the pilot study including pilot feedback supporting the potential for increased automation and a single throttle control. Detailed recommendations are provided for a novel single throttle control and additional pilot controls to support selection of engines during start, shutdown, and engine troubleshooting procedures. This design deviates significantly from current practice of providing throttles for each propulsion engine. Engine display recommendations are provided based on pilot feedback during a guided post-evaluation interview. Battery state of charge and battery health display recommendations were collected from all airline crews. The simplified engine displays design was rated excellent as measured with a usability scale. Quantitative metrics include airspeed tracking, time to complete checklists, time to make diversion decisions and the quality of the diversion decision. Recommendations for future studies are documented with supporting research and current observations about upcoming flight deck certifications.

autothrottle↗

NASA NTRS1977

Radiographic amplifier screens: Fabrication process and characteristics

The fabrication process and transfer characteristics for solid state radiographic image transducers (radiographic amplifier screens) is described. These screens were developed for use in real time nondestructive evaluation procedures that require large format radiographic images with contrast and resolution capabilities unavailable with conventional fluoroscopic screens. This work was directed toward screens usable for inmotion, on-line radiographic inspection by means of closed circuit television.

Szepesi, Z. P.↗

NASA NTRS1993

The use of analytical models in human-computer interface design

Recently, a large number of human-computer interface (HCI) researchers have investigated building analytical models of the user, which are often implemented as computer models. These models simulate the cognitive processes and task knowledge of the user in ways that allow a researcher or designer to estimate various aspects of an interface's usability, such as when user errors are likely to occur. This information can lead to design improvements. Analytical models can supplement design guidelines by providing designers rigorous ways of analyzing the information-processing requirements of specific tasks (i.e., task analysis). These models offer the potential of improving early designs and replacing some of the early phases of usability testing, thus reducing the cost of interface design. This paper describes some of the many analytical models that are currently being developed and evaluates the usefulness of analytical models for human-computer interface design. This paper will focus on computational, analytical models, such as the GOMS model, rather than less formal, verbal models, because the more exact predictions and task descriptions of computational models may be useful to designers. The paper also discusses some of the practical requirements for using analytical models in complex design organizations such as NASA.

Gugerty, Leo↗

NASA NTRS1994

Advanced End-to-end Simulation for On-board Processing (AESOP)

Developers of data compression algorithms typically use their own software together with commercial packages to implement, evaluate and demonstrate their work. While convenient for an individual developer, this approach makes it difficult to build on or use another's work without intimate knowledge of each component. When several people or groups work on different parts of the same problem, the larger view can be lost. What's needed is a simple piece of software to stand in the gap and link together the efforts of different people, enabling them to build on each other's work, and providing a base for engineers and scientists to evaluate the parts as a cohesive whole and make design decisions. AESOP (Advanced End-to-end Simulation for On-board Processing) attempts to meet this need by providing a graphical interface to a developer-selected set of algorithms, interfacing with compiled code and standalone programs, as well as procedures written in the IDL and PV-Wave command languages. As a proof of concept, AESOP is outfitted with several data compression algorithms integrating previous work on different processors (AT&T DSP32C, TI TMS320C30, SPARC). The user can specify at run-time the processor on which individual parts of the compression should run. Compressed data is then fed through simulated transmission and uncompression to evaluate the effects of compression parameters, noise and error correction algorithms. The following sections describe AESOP in detail. Section 2 describes fundamental goals for usability. Section 3 describes the implementation. Sections 4 through 5 describe how to add new functionality to the system and present the existing data compression algorithms. Sections 6 and 7 discuss portability and future work.

Mazer, Alan S.↗

NASA NTRS2001

Electronic Nose: Evaluation of Kamina Prototype Unit

The Kamina, Sam and Cyranose electronic nose systems were evaluated and partially trained. Much work was performed on the Kamina as it has the ability to respond to low (less than 10 ppb) concentrations of hydrazine compounds. We were able to tell the difference between Hydrazine (Hz) and Monomethylhydrazine (MMH) in standard clean humid air. We were able to detect MMH in reduced pressure (1/3 atm) at about 250 ppb, however the training set was to far from the real situation to be useful now. Various engineering and usability aspects of both the noses was noted, especially the software. One serious physical engineering flaw was remedied in the Kamina system. A gas flow manifold was created for the Sam system. Different chips were evaluated for the Kamina system. It is still unclear if they can be exchanged without retraining the software.The Sam Detect commercial unit was evaluated for solvent detection and evaluation. It was able to successfully identify some solvents. The Cyranose, was observed and evaluated for two days. It has the ability to detect gasses in the 100 parts per million level but not the 10 parts per billion level. It is very sensitive to humidity changes; there is software to partially handle this.

Schattke, Nathan↗