Search NASASearch

SEARCH · Search NASA

Results for “Human Computer Interaction”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Monitoring Outpatient Care

Each year, health care costs for managing chronically ill patients increase as the life expectancy of Americans continues to grow. To handle this situation, many hospitals, doctors practices, and home care providers are turning to disease management, a system of coordinated health care interventions and communications, to improve outpatient care. By participating in daily monitoring programs, patients with congestive heart failure, chronic obstructive pulmonary disease, diabetes, and other chronic conditions requiring significant self-care are facing fewer emergency situations and hospitalizations. Cybernet Medical, a division of Ann Arbor, Michigan-based Cybernet Systems Corporation, is using the latest communications technology to augment the ways health care professionals monitor and assess patients with chronic diseases, while at the same time simplifying the patients interaction with technology. Cybernet s newest commercial product for this purpose evolved from research funded by NASA, the National Institute of Mental Health, and the Advanced Research Projects Agency. The research focused on the physiological assessment of astronauts and soldiers, human performance evaluation, and human-computer interaction. Cybernet Medical's MedStar Disease Management Data Collection System is an affordable, widely deployable solution for improving in-home-patient chronic disease management. The system's battery-powered and portable interface device collects physiological data from off-the-shelf instruments.

Source record

Developing Visualization Techniques for Semantics-based Information Networks

Information systems incorporating complex network structured information spaces with a semantic underpinning - such as hypermedia networks, semantic networks, topic maps, and concept maps - are being deployed to solve some of NASA s critical information management problems. This paper describes some of the human interaction and navigation problems associated with complex semantic information spaces and describes a set of new visual interface approaches to address these problems. A key strategy is to leverage semantic knowledge represented within these information spaces to construct abstractions and views that will be meaningful to the human user. Human-computer interaction methodologies will guide the development and evaluation of these approaches, which will benefit deployed NASA systems and also apply to information systems based on the emerging Semantic Web.

Keller, Richard M.

Multiuser Collaboration with Networked Mobile Devices

In this paper we describe a multiuser collaboration infrastructure that enables multiple mission scientists to remotely and collaboratively interact with visualization and planning software, using wireless networked personal digital assistants(PDAs) and other mobile devices. During ground operations of planetary rover and lander missions, scientists need to meet daily to review downlinked data and plan science activities. For example, scientists use the Science Activity Planner (SAP) in the Mars Exploration Rover (MER) mission to visualize downlinked data and plan rover activities during the science meetings [1]. Computer displays are projected onto large screens in the meeting room to enable the scientists to view and discuss downlinked images and data displayed by SAP and other software applications. However, only one person can interact with the software applications because input to the computer is limited to a single mouse and keyboard. As a result, the scientists have to verbally express their intentions, such as selecting a target at a particular location on the Mars terrain image, to that person in order to interact with the applications. This constrains communication and limits the returns of science planning. Furthermore, ground operations for Mars missions are fundamentally constrained by the short turnaround time for science and engineering teams to process and analyze data, plan the next uplink, generate command sequences, and transmit the uplink to the vehicle [2]. Therefore, improving ground operations is crucial to the success of Mars missions. The multiuser collaboration infrastructure enables users to control software applications remotely and collaboratively using mobile devices. The infrastructure includes (1) human-computer interaction techniques to provide natural, fast, and accurate inputs, (2) a communications protocol to ensure reliable and efficient coordination of the input devices and host computers, (3) an application-independent middleware that maintains the states, sessions, and interactions of individual users of the software applications, (4) an application programming interface to enable tight integration of applications and the middleware. The infrastructure is able to support any software applications running under the Windows or Unix platforms. The resulting technologies not only are applicable to NASA mission operations, but also useful in other situations such as design reviews, brainstorming sessions, and business meetings, as they can benefit from having the participants concurrently interact with the software applications (e.g., presentation applications and CAD design tools) to illustrate their ideas and provide inputs.

ground operations

Enabling a Voice Management System for Space Applications

The sustainable missions beyond Low Earth Orbit (LEO) envisioned for NASA’s Artemis program will require autonomous capabilities. Moreover, Artemis mission crews will need a means to efficiently interact with a spacecraft’s autonomous systems. This interaction can be facilitated by voice and speech communications because voice-based controls enable users to interact hands- and eyes-free, allowing the user to better focus on critical tasks. The goal of our project was to explore the knowledge and technology needed to successfully design effective Voice User Interfaces (VUIs) for autonomous systems utilizing Human Centered Design (HCD) principles. The focus of the human factors’ aspect of engineering, pays close attention to psychological and physiological principles in the development of autonomous crew operation systems. A main objective was to understand how a crew member, through voice interaction, could efficiently and intuitively communicate with a notional autonomous vehicle system manager. This project was a part of the NASA Moon to Mars eXploration Systems and Habitation (M2M X-Hab) 2020 Academic Innovation Challenge. The work from the BLiSS Team, at the University of Michigan, resulted in the design of a system persona, Diego, to which an astronaut may quickly build trust with autonomous systems, to alleviate known stressors on mental health expected during long duration space missions. Optimal software to facilitate integration of the system persona into a reference Lunar orbiting Gateway station was defined. Additionally, a Speech to Text (STT) system and a Graphical User Interface (GUI) that could be implemented in future missions was developed on an Internet of Things (IOT) platform. The Voice User Interface (VUI) design for the M2M X-Hab 2020 project leveraged previous technology developed by the BLiSS team to incorporate a voice-based interface into NASA’s Platform for Autonomous Systems (NPAS) software. This required technologies to convert voice to text, conduct semantic interpretations, and convert responses from the autonomous system to text and to speech; additionally, the spacecraft background noise environment was assessed, a noise mitigation technique was developed, and a relatable personality for the autonomous system was developed in order to facilitate human-like conversations. The success of our effort was largely due to the diversity of the team that included expertise in Space Systems Engineering, Human Computer Interaction, Aerospace Engineering, Computer Science, Biomedical Engineering, and Applied Physics. The diverse perspectives fostered elaborate discussions, resulting in the conception of three main subsystems: (1) User-System, (2) NPAS-System, and (3) Environment-System. The VUI was unique and had to be efficient and intuitive. For this project, 5 subteams were formed, each with a separate objective, Voice Design team, Background Noise Mitigation team, Software Integration team and Graphical User Interface team. The BLiSS team crafted a personality for the VUI to enable human-like conversation and drive user adoption and trust. User surveys were completed and used to help determine the required VUI system personality traits by capturing perspectives and expectations of prospective “Artemis Generation Astronauts”. To further simulate human-like conversations, the system had to be able to quickly interpret user speech and be able to integrate with NASA’s NPAS platform for quick and reliable information transfer. The outcomes of our research were: (1) a working prototype user interface, that is compatible with NASA’s NPAS platform; (2) software that demonstrates the ability of the VUI system to interpret user requests and respond appropriately; (3) the capability to implement fully expanded conversations between user and system using intuitive communication in four request categories; and (4) software and hardware recommendations that optimize the system’s ability to operate in a noisy environment. Our research has laid the foundation for the development of VUI’s for autonomy, and provides a baseline for future VUI developments.

Voice user interface

Large language model evaluation for high–performance computing software development

We apply AI-assisted large language model (LLM) capabilities of GPT-3 targeting high-performance computing (HPC) kernels for (i) code generation, and (ii) auto-parallelization of serial code in C ++, Fortran, Python and Julia. Our scope includes the following fundamental numerical kernels: AXPY, GEMV, GEMM, SpMV, Jacobi Stencil, and CG, and language/programming models: (1) C++ (e.g., OpenMP [including offload], OpenACC, Kokkos, SyCL, CUDA, and HIP), (2) Fortran (e.g., OpenMP [including offload] and OpenACC), (3) Python (e.g., numpy, Numba, cuPy, and pyCUDA), and (4) Julia (e.g., Threads, CUDA.jl, AMDGPU.jl, and KernelAbstractions.jl). Kernel implementations are generated using GitHub Copilot capabilities powered by the GPT-based OpenAI Codex available in Visual Studio Code given simple + + prompt variants. To quantify and compare the generated results, we propose a proficiency metric around the initial 10 suggestions given for each prompt. For auto-parallelization, we use ChatGPT interactively giving simple prompts as in a dialogue with another human including simple “prompt engineering” follow ups. Results suggest that correct outputs for C++ correlate with the adoption and maturity of programming models. For example, OpenMP and CUDA score really high, whereas HIP is still lacking. We found that prompts from either a targeted language such as Fortran or the more general-purpose Python can benefit from adding language keywords, while Julia prompts perform acceptably well for its Threads and CUDA.jl programming models. Finally, we expect to provide an initial quantifiable point of reference for code generation in each programming model using a state-of-the-art LLM. Overall, understanding the convergence of LLMs, AI, and HPC is crucial due to its rapidly evolving nature and how it is redefining human-computer interactions.

97 MATHEMATICS AND COMPUTING

2015 Space Human Factors Engineering Standing Review Panel

The 2015 Space Human Factors Engineering (SHFE) Standing Review Panel (from here on referred to as the SRP) met for a site visit in Houston, TX on December 2 - 3, 2015. The SRP reviewed the updated research plans for the Risk of Inadequate Design of Human and Automation/Robotic Integration (HARI Risk), the Risk of Inadequate Human-Computer Interaction (HCI Risk), and the Risk of Inadequate Mission, Process and Task Design (MPTask Risk). The SRP also received a status update on the Risk of Incompatible Vehicle/Habitat Design (Hab Risk) and the Risk of Performance Errors Due to Training Deficiencies (Train Risk). The SRP is pleased with the progress and responsiveness of the SHFE team. The presentations were much improved this year. The SRP is also pleased with the human-centered design approach. Below are some of the more extensive comments from the SRP. We have also made comments in each section concerning gaps/tasks in each. The comments below reflect more significant changes that impact more than just one particular section.

Steinberg, Susan

OFMspert: An architecture for an operator's associate that evolves to an intelligent tutor

With the emergence of new technology for both human-computer interaction and knowledge-based systems, a range of opportunities exist which enhance the effectiveness and efficiency of controllers of high-risk engineering systems. The design of an architecture for an operator's associate is described. This associate is a stand-alone model-based system designed to interact with operators of complex dynamic systems, such as airplanes, manned space systems, and satellite ground control systems in ways comparable to that of a human assistant. The operator function model expert system (OFMspert) architecture and the design and empirical validation of OFMspert's understanding component are described. The design and validation of OFMspert's interactive and control components are also described. A description of current work in which OFMspert provides the foundation in the development of an intelligent tutor that evolves to an assistant, as operator expertise evolves from novice to expert, is provided.

Mitchell, Christine M.

The NASA Augmented/Virtual Reality Lab: The State of the Art at KSC

The NASA Augmented Virtual Reality (AVR) Lab at Kennedy Space Center is dedicated to the investigation of Augmented Reality (AR) and Virtual Reality (VR) technologies, with the goal of determining potential uses of these technologies as human-computer interaction (HCI) devices in an aerospace engineering context. Begun in 2012, the AVR Lab has concentrated on commercially available AR and VR devices that are gaining in popularity and use in a number of fields such as gaming, training, and telepresence. We are working with such devices as the Microsoft Kinect, the Oculus Rift, the Leap Motion, the HTC Vive, motion capture systems, and the Microsoft Hololens. The focus of our work has been on human interaction with the virtual environment, which in turn acts as a communications bridge to remote physical devices and environments which the operator cannot or should not control or experience directly. Particularly in reference to dealing with spacecraft and the oftentimes hazardous environments they inhabit, it is our hope that AR and VR technologies can be utilized to increase human safety and mission success by physically removing humans from those hazardous environments while virtually putting them right in the middle of those environments.

Little, William

Intelligent systems and advanced user interfaces for design, operation, and maintenance of command management systems

Historically, command management systems (CMS) have been large and expensive spacecraft-specific software systems that were costly to build, operate, and maintain. Current and emerging hardware, software, and user interface technologies may offer an opportunity to facilitate the initial formulation and design of a spacecraft-specific CMS as well as to develop a more generic CMS system. New technologies, in addition to a core CMS common to a range of spacecraft, may facilitate the training and enhance the efficiency of CMS operations. Current mission operations center (MOC) hardware and software include Unix workstations, the C/C++ programming languages, and an X window interface. This configuration provides the power and flexibility to support sophisticated and intelligent user interfaces that exploit state-of-the-art technologies in human-machine interaction, artificial intelligence, and software engineering. One of the goals of this research is to explore the extent to which technologies developed in the research laboratory can be productively applied in a complex system such as spacecraft command management. Initial examination of some of these issues in CMS design and operation suggests that application of technologies such as intelligent planning, case-based reasoning, human-machine systems design and analysis tools (e.g., operator and designer models), and human-computer interaction tools (e.g., graphics, visualization, and animation) may provide significant savings in the design, operation, and maintenance of the CMS for a specific spacecraft as well as continuity for CMS design and development across spacecraft. The first six months of this research saw a broad investigation by Georgia Tech researchers into the function, design, and operation of current and planned command management systems at Goddard Space Flight Center. As the first step, the researchers attempted to understand the current and anticipated horizons of command management systems at Goddard. Preliminary results are given on CMS commonalities and causes of low re-use, and methods are proposed to facilitate increased re-use.

Potter, William J.

Information for the user in design of intelligent systems

Recommendations are made for improving intelligent system reliability and usability based on the use of information requirements in system development. Information requirements define the task-relevant messages exchanged between the intelligent system and the user by means of the user interface medium. Thus, these requirements affect the design of both the intelligent system and its user interface. Many difficulties that users have in interacting with intelligent systems are caused by information problems. These information problems result from the following: (1) not providing the right information to support domain tasks; and (2) not recognizing that using an intelligent system introduces new user supervisory tasks that require new types of information. These problems are especially prevalent in intelligent systems used for real-time space operations, where data problems and unexpected situations are common. Information problems can be solved by deriving information requirements from a description of user tasks. Using information requirements embeds human-computer interaction design into intelligent system prototyping, resulting in intelligent systems that are more robust and easier to use.

Malin, Jane T.

Application Reuse Library for Software, Requirements, and Guidelines

Better designs are needed for expert systems and other operations automation software, for more reliable, usable and effective human support. A prototype computer-aided Application Reuse Library shows feasibility of supporting concurrent development and improvement of advanced software by users, analysts, software developers, and human-computer interaction experts. Such a library expedites development of quality software, by providing working, documented examples, which support understanding, modification and reuse of requirements as well as code. It explicitly documents and implicitly embodies design guidelines, standards and conventions. The Application Reuse Library provides application modules with Demo-and-Tester elements. Developers and users can evaluate applicability of a library module and test modifications, by running it interactively. Sub-modules provide application code and displays and controls. The library supports software modification and reuse, by providing alternative versions of application and display functionality. Information about human support and display requirements is provided, so that modifications will conform to guidelines. The library supports entry of new application modules from developers throughout an organization. Example library modules include a timer, some buttons and special fonts, and a real-time data interface program. The library prototype is implemented in the object-oriented G2 environment for developing real-time expert systems.

Malin, Jane T.

Mode Transitions in Glass Cockpit Aircraft: Results of a Field Study

One consequence of increased levels of automation in complex control systems is the presence of modes. A mode is a particular configuration of a control system that defines how human command inputs are interpreted. In complex systems, modes also often determine a specific allocation of control authority between the human and automated systems. Even in simple static devices (e.g., electronic watches, word processors), the presence of modes has been found to cause problems in either-the acquisition or production of skilled performance. Many of these problems arise due to the fact that the selection of a mode causes device behavior to be mediated by hidden internal state information. For these simple systems, many of these interaction problems can be solved by the design of appropriate feedback to communicate internal state information to the human operator. In complex dynamic systems, however, the design issues associated with modes seem to trancend the problem of merely communicating internal state information via displayed feedback. In complex supervisory control systems (e.g., aircraft, spacecraft, military command and control), a key function of modes is the selection of a particular configuration of control authority between the human operator and automated control systems. One mode may result in full manual control, another may result in a mix of manual and automatic control, while a third may result in full automatic control over the entire system. The human operator selects an appropriate mode as a function of current goals, operating conditions, and operating procedures. Thus, the operator is put in a position of essentially trying to control two coupled dynamic systems: the target system itself, and also a highly complex suite of automation controlling the target system. From a historical perspective, it should probably not come as a surprise that very little information is available to guide the design of mode-oriented control systems. The topic of function allocation (i.e., the proper division of control authority among human and computer) has a long history in human-machine systems research. Although this research has produced some relevant guidelines, a design approach capable of defining appropriate allocations of control function between the human and automation is not yet available. As a result, the function allocation decision itself has been allocated to the operator, to be performed in real-time, in the operation of mode-oriented control systems. A variety of documented aircraft accidents and incidents suggest that the real-time selection and monitoring of control modes is a weak link in the effective operation of complex supervisory control systems. Research in human-machine systems and human-computer interaction has barely scraped the surface of the problem of understanding how operators manage this task.The purpose of this paper is to present the results of a field study which examined how operators manage mode selection in a complex supervisory control system. Data on mode engagements using the Boeing B757/767 auto-flight system were collected during approach and descent into four major airports in the East Coast of the United States. Protocols documenting mode selection, automatic mode changes, pilot actions, quantitative records of flight-path variables, and verbal reports during and after mode engagements were collected by an observer from the jumpseat. Observations were conducted on two typical trips between three airports. Each trip was be replicated 11 times, which yielded a total of 22 trips and 66 legs on which data were collected. All data collected concerned the same flight numbers, and therefore, the same time of day, same type of aircraft, and identical operational environments (e.g., ATC facilities, weather patterns, traffic flow etc.)

Degani, Asaf

Radsource Mr: Mixed Reality Planning Tool For Radioactive Recovery

The RadSource MR system leverages Meta Quest 3's advanced mixed reality capabilities to create a comprehensive spatial planning platform for end-of-life sealed radioactive source recovery operations. The application utilizes the Quest 3's high-resolution passthrough cameras and spatial mapping algorithms to generate accurate 3D environmental models. Core technical components include: (1) Real-time spatial measurement algorithms calculating distances, angles, slopes, and surface areas with sub-centimeter accuracy; (2) Virtual object placement system allowing users to position digital representations of recovery equipment (trailers, containment vessels, protective barriers) within the real environment; (3) Voice recording and annotation system for hands-free documentation in protective equipment; (4) 3D mesh capture and storage capabilities for post-operation analysis and regulatory documentation. (5) Procedure documentation is available for viewing in Mixed Reality, providing an innovative and convenient way to access the information during pre-visit and inspection activities. (6) Support for screen capture for the view for real world and virtual objects together to use it later for planning. The system integrates computer vision techniques for environmental understanding, spatial mathematics for precise measurements, and human-computer interaction principles optimized for hazardous environment operations. Data persistence allows teams to save and share planning sessions across multiple stakeholders while maintaining operational security requirements.

Khadka, Rajiv [Idaho National Laboratory (INL), Id

Multimodal user input to supervisory control systems - Voice-augmented keyboard

The use of a voice-augmented keyboard input modality is evaluated in a supervisory control application. An implementation of voice recognition technology in supervisory control is proposed: voice is used to request display pages, while the keyboard is used to input system reconfiguration commands. Twenty participants controlled GT-MSOCC, a high-fidelity simulation of the operator interface to a NASA ground control system, via a workstation equipped with either a single keyboard or a voice-augmented keyboard. Experimental results showed that in all cases where significant performance differences occurred, performance with the voice-augmented keyboard modality was inferior to and had greater variance than the keyboard-only modality. These results suggest that current moderately priced voice recognition systems are an inappropriate human-computer interaction technology in supervisory control systems.

Mitchell, Christine M.

OFMspert - Inference of operator intentions in supervisory control using a blackboard architecture

The authors proposes an architecture for an expert system that can function as an operator's associate in the supervisory control of a complex dynamic system. Called OFMspert (operator function model (OFM) expert system), the architecture uses the operator function modeling methodology as the basis for the design. The authors put emphasis on the understanding capabilities, i.e., the intent referencing property, of an operator's associate. The authors define the generic structure of OFMspert, particularly those features that support intent inferencing. They also describe the implementation and validation of OFMspert in GT-MSOCC (Georgia Tech-Multisatellite Operations Control Center), a laboratory domain designed to support research in human-computer interaction and decision aiding in complex, dynamic systems.

Jones, Patricia S.

Model-based displays for satellite ground control

With the emergence of new technology for both human-computer interaction and knowledge-based systems, a range of opportunities exists to enhance the effectiveness and efficiency of satellite ground controllers. The use of models of operator function to represent operator activity in the context of changing system events and operator functions is illustrated. Although there are many models, the operator function model (OFM) was used in this research. In addition to representing operator activities, the OFM can be used to design 'intelligent' operator displays and, in real time, control the displayed information so that the operator has the appropriate information, at the appropriate time, and at the appropriate level of abstraction. The OFM was demonstrated in the context of a NASA Goddard Space Flight Center satellite ground control system. To evaluate the effectiveness of the model-based workstation, an experiment was conducted to compare system performance with a conventional operator workstation versus the model based workstation.

Mitchell, Christine M.

Developing the human-computer interface for Space Station Freedom

For the past two years, the Human-Computer Interaction Laboratory (HCIL) at the Johnson Space Center has been involved in prototyping and prototype reviews of in support of the definition phase of the Space Station Freedom program. On the Space Station, crew members will be interacting with multi-monitor workstations where interaction with several displays at one time will be common. The HCIL has conducted several experiments to begin to address design issues for this complex system. Experiments have dealt with design of ON/OFF indicators, the movement of the cursor across multiple monitors, and the importance of various windowing capabilities for users performing multiple tasks simultaneously.

Holden, Kritina L.

Hypermedia = hypercommunication

New hardware and software technology gave application designers the freedom to use new realism in human computer interaction. High-quality images, motion video, stereo sound and music, speech, touch, gesture provide richer data channels between the person and the machine. Ultimately, this will lead to richer communication between people with the computer as an intermediary. The whole point of hyper-books, hyper-newspapers, virtual worlds, is to transfer the concept and relationships, the 'data structure', from the mind of creator to that of user. Some of the characteristics of this rich information channel are discussed, and some examples are presented.

Laff, Mark R.