Search NASA⌕ Search

SEARCH · Search NASA

Results for “human computer interaction”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

Mission Planning for Robotic and Human Exploration

Mission planning for spaceflight is a complex challenge for ground operators managing robotic or human missions. It requires integrating the needs of various disciplines and creating an effective, efficient, and safe plan to execute. NASA uses specialized tools to meet planning requirements and constraints. This talk will provide an overview of spaceflight mission planning as well as the software tools the Human-Computer Interaction Group has built to support scheduling and planning. Also, it identifies the future scheduling and planning challenges for human Mars missions and how we leverage Earth Analogs to learn about Mars operations.

Scheduling and planning↗
NASA NTRS2019

Reusing Data and Metadata to Create New Metadata Through Machine-Learning & Other Programmatic Methods

Recent improvements in natural language processing (NLP) enable metadata to be created programmatically from reused original metadata or even the dataset itself. Transfer-learning applied to NLP has greatly improved performance and reduced training data requirements. In this talk, we’ll compare machine-generated metadata to human-generated metadata and discuss characteristics of metadata and data archives that affect suitability for machine-learning reuse of metadata. Where as human-generated metadata is often populated once, populated from the perspective of data supplier, populated by many individuals with different words for the same thing, and limited in length, machine-generated metadata can be updated any number of times, generated from the perspective of any user, constrained to a standardized set of terms that can be evolved over time, and be any length required. Machine-learning generated metadata offers benefits but also additional needs in terms of version control, process transparency, human-computer interaction, and IT requirements. As a successful example, we’ll discuss how a dataset of abstracts and associated human-tagged keywords from a standardized list of several thousand keywords were used to create a machine-learning model that predicted keyword metadata for open-source code projects on code.nasa.gov. We’ll also discuss a less successful example from data.nasa.gov to show how data archive architecture and characteristics of initial metadata can be strong controls on how easy it is to leverage programmatic methods to reuse metadata to create additional metadata.

Gosses, Justin↗
DOE OSTI2025

VISION: a modular AI assistant for natural human-instrument interaction at scientific user facilities

Scientific user facilities, such as synchrotron beamlines, are equipped with a wide array of hardware and software tools that require a codebase for human-computer-interaction. This often necessitates developers to be involved to establish connection between users/researchers and the complex instrumentation. The advent of generative AI presents an opportunity to bridge this knowledge gap, enabling seamless communication and efficient experimental workflows. Here we present a modular architecture for the Virtual Scientific Companion by assembling multiple AI-enabled cognitive blocks that each scaffolds large language models (LLMs) for a specialized task. With VISION, we performed LLM-based operation on the beamline workstation with low latency and demonstrated the first voice-controlled experiment at an x-ray scattering beamline. The modular and scalable architecture allows for easy adaptation to new instruments and capabilities. Development on natural language-based scientific experimentation is a building block for an impending future where a science exocortex—a synthetic extension to the cognition of scientists—may radically transform scientific practice and discovery.

36 MATERIALS SCIENCE↗
NASA NTRS2017

Playbook Data Analysis Tool: Collecting Interaction Data from Extremely Remote Users

Typically, user tests for software tools are conducted in person. At NASA, the users may be located at the bottom of the ocean in a pressurized habitat, above the atmosphere in the International Space Station, or in an isolated capsule on a simulated asteroid mission. The Playbook Data Analysis Tool (P-DAT) is a human-computer interaction (HCI) evaluation tool that the NASA Ames HCI Group has developed to record user interactions with Playbook, the group's existing planning-and-execution software application. Once the remotely collected user interaction data makes its way back to Earth, researchers can use P-DAT for in-depth analysis. Since a critical component of the Playbook project is to understand how to develop more intuitive software tools for astronauts to plan in space, P-DAT helps guide us in the development of additional easy-to-use features for Playbook, informing the design of future crew autonomy tools.P-DAT has demonstrated the capability of discreetly capturing usability data in amanner that is transparent to Playbook’s end-users. In our experience, P-DAT data hasalready shown its utility, revealing potential usability patterns, helping diagnose softwarebugs, and identifying metrics and events that are pertinent to Playbook usage aswell as spaceflight operations. As we continue to develop this analysis tool, P-DATmay yet provide a method for long-duration, unobtrusive human performance collectionand evaluation for mission controllers back on Earth and researchers investigatingthe effects and mitigations related to future human spaceflight performance.

in-flight monitoring↗
NASA NTRS2012

Interpersonal Biocybernetics: Connecting Through Social Psychophysiology

One embodiment of biocybernetic adaptation is a human-computer interaction system designed such that physiological signals modulate the effect that control of a task by other means, usually manual control, has on performance of the task. Such a modulation system enables a variety of human-human interactions based upon physiological self-regulation performance. These interpersonal interactions may be mixes of competition and cooperation for simulation training and/or videogame entertainment

Pope, Alan T.↗
NASA NTRSDate not supplied

Enabling a Voice Management System for Space Applications, Design and Software Development

Sustainable missions, beyond low Earth orbit, will require autonomous capabilities in order to achieve NASA’s Artemis program objectives. Correspondingly, the crew must have a means to efficiently interact with these autonomous systems; this can be facilitated via voice and speech communications. Voice-based controls enable the user to access autonomous systems hands-free/eyes-free, allowing the user to better focus on critical tasks. The goal of this project was to explore the knowledge and technology needed to successfully design effective voice interfaces for autonomous systems. The main objective was to understand how a crew member, through voice interaction, could most efficiently and intuitively communicate with a notional autonomous vehicle system manager. This project leveraged prior research conducted by the University of Michigan’s Bioastronautics and Life Support System (BLiSS) team as part of a NASA Moon to Mars eXploration Systems and Habitation (M2M X-Hab) 2020 Academic Innovation Challenge. The X-Hab 2020 work from the BliSS Team resulted in an intuitive graphical user interface/user experience that was built on an Internet of Things (IOT) platform. The Voice User Interface (VUI) design for the M2M X-Hab 2021 project leveraged this technology and incorporated a voice-based assistant and NASA’s Platform for Autonomous Systems (NPAS) software. This required technologies to convert voice to text, conduct semantic interpretations, and convert responses from the autonomous system to text and to speech; additionally, the background noise environment of spacecraft was assessed, and a relatable personality for the autonomous system to facilitate human-like conversations was created. This work’s success was largely due to the diverse team that included expertise in Space Systems Engineering, Human Computer Interaction, Aerospace Engineering, Computer Science, Biomedical Engineering, and Applied Physics. The differing perspectives fostered elaborate discussions, resulting in the conception of three main interactions: (1) User-System, (2) NPAS-System, and (3) Environment-System. The system developed, i.e. the VUI, had to be unique, efficient, and intuitive; thus, the team crafted a personality for the system to enable human-like conversation. User surveys sent to students and young professionals were used to help determine these personality traits by capturing perspectives and expectations of the “Artemis Generation Astronauts”. To further simulate human-like conversations, the system had to be able to quickly interpret user speech and be able to integrate with NASA’s NPAS system for quick and reliable information transfer. Results of this research include (1) a working prototype user interface, that is compatible with NASA’s NPAS system; (2) software that demonstrates the ability to interpret user requests and respond appropriately; (3) the capability to implement fully expanded conversations between user and system using intuitive communication in four request categories; and (4) software and hardware recommendations that optimize the system’s ability to operate, i.e. be heard, in a noisy environment. The technologies chosen for this project’s demonstrations included the following: Raspberry Pi, RASA, Mozilla Deep Speech, Coqui, RTX Voice and Adobe XD. This work has laid the foundation for the development of VUI’s used for autonomy, and is intended to provide guidance for future VUI development.

Tara Vega↗
NASA NTRS2003

Monitoring Outpatient Care

Each year, health care costs for managing chronically ill patients increase as the life expectancy of Americans continues to grow. To handle this situation, many hospitals, doctors practices, and home care providers are turning to disease management, a system of coordinated health care interventions and communications, to improve outpatient care. By participating in daily monitoring programs, patients with congestive heart failure, chronic obstructive pulmonary disease, diabetes, and other chronic conditions requiring significant self-care are facing fewer emergency situations and hospitalizations. Cybernet Medical, a division of Ann Arbor, Michigan-based Cybernet Systems Corporation, is using the latest communications technology to augment the ways health care professionals monitor and assess patients with chronic diseases, while at the same time simplifying the patients interaction with technology. Cybernet s newest commercial product for this purpose evolved from research funded by NASA, the National Institute of Mental Health, and the Advanced Research Projects Agency. The research focused on the physiological assessment of astronauts and soldiers, human performance evaluation, and human-computer interaction. Cybernet Medical's MedStar Disease Management Data Collection System is an affordable, widely deployable solution for improving in-home-patient chronic disease management. The system's battery-powered and portable interface device collects physiological data from off-the-shelf instruments.

Source record↗
NASA NTRS2003

Developing Visualization Techniques for Semantics-based Information Networks

Information systems incorporating complex network structured information spaces with a semantic underpinning - such as hypermedia networks, semantic networks, topic maps, and concept maps - are being deployed to solve some of NASA s critical information management problems. This paper describes some of the human interaction and navigation problems associated with complex semantic information spaces and describes a set of new visual interface approaches to address these problems. A key strategy is to leverage semantic knowledge represented within these information spaces to construct abstractions and views that will be meaningful to the human user. Human-computer interaction methodologies will guide the development and evaluation of these approaches, which will benefit deployed NASA systems and also apply to information systems based on the emerging Semantic Web.

Keller, Richard M.↗
NASA NTRS2006

Multiuser Collaboration with Networked Mobile Devices

In this paper we describe a multiuser collaboration infrastructure that enables multiple mission scientists to remotely and collaboratively interact with visualization and planning software, using wireless networked personal digital assistants(PDAs) and other mobile devices. During ground operations of planetary rover and lander missions, scientists need to meet daily to review downlinked data and plan science activities. For example, scientists use the Science Activity Planner (SAP) in the Mars Exploration Rover (MER) mission to visualize downlinked data and plan rover activities during the science meetings [1]. Computer displays are projected onto large screens in the meeting room to enable the scientists to view and discuss downlinked images and data displayed by SAP and other software applications. However, only one person can interact with the software applications because input to the computer is limited to a single mouse and keyboard. As a result, the scientists have to verbally express their intentions, such as selecting a target at a particular location on the Mars terrain image, to that person in order to interact with the applications. This constrains communication and limits the returns of science planning. Furthermore, ground operations for Mars missions are fundamentally constrained by the short turnaround time for science and engineering teams to process and analyze data, plan the next uplink, generate command sequences, and transmit the uplink to the vehicle [2]. Therefore, improving ground operations is crucial to the success of Mars missions. The multiuser collaboration infrastructure enables users to control software applications remotely and collaboratively using mobile devices. The infrastructure includes (1) human-computer interaction techniques to provide natural, fast, and accurate inputs, (2) a communications protocol to ensure reliable and efficient coordination of the input devices and host computers, (3) an application-independent middleware that maintains the states, sessions, and interactions of individual users of the software applications, (4) an application programming interface to enable tight integration of applications and the middleware. The infrastructure is able to support any software applications running under the Windows or Unix platforms. The resulting technologies not only are applicable to NASA mission operations, but also useful in other situations such as design reviews, brainstorming sessions, and business meetings, as they can benefit from having the participants concurrently interact with the software applications (e.g., presentation applications and CAD design tools) to illustrate their ideas and provide inputs.

ground operations↗
DOE OSTI2023

Visualization and Decision Making Design Under Uncertainty

This report documents the program and the outcomes of Dagstuhl Seminar 22331 "Visualization and Decision Making Design Under Uncertainty". The seminar brought together 33 researchers and practitioners from different domains concerned with visualization and decision making under uncertainty including visualization, visual analytics, human-computer interaction, artificial intelligence, climate research, geography and geology. The programme was organized in two parts: In the first part which lasted two days, participants gave short talks where they discussed current practices and the uncertainty visualization challenges they encountered in their own research. At the end of day two, participants brainstormed collectively around the main uncertainty visualization research challenges across domains and applications. In the second part, participants voted for the following three main challenges they wished to discuss for the remainder of the seminar (one and a half days): applications, human-centered uncertainty visualization, a design process for uncertainty visualization. Thus three break-out groups were formed to discuss these challenges. Abstracts for the individual talks and the break-out group activities are included in this report.

decision making↗
NASA NTRSDate not supplied

Enabling a Voice Management System for Space Applications

The sustainable missions beyond Low Earth Orbit (LEO) envisioned for NASA’s Artemis program will require autonomous capabilities. Moreover, Artemis mission crews will need a means to efficiently interact with a spacecraft’s autonomous systems. This interaction can be facilitated by voice and speech communications because voice-based controls enable users to interact hands- and eyes-free, allowing the user to better focus on critical tasks. The goal of our project was to explore the knowledge and technology needed to successfully design effective Voice User Interfaces (VUIs) for autonomous systems utilizing Human Centered Design (HCD) principles. The focus of the human factors’ aspect of engineering, pays close attention to psychological and physiological principles in the development of autonomous crew operation systems. A main objective was to understand how a crew member, through voice interaction, could efficiently and intuitively communicate with a notional autonomous vehicle system manager. This project was a part of the NASA Moon to Mars eXploration Systems and Habitation (M2M X-Hab) 2020 Academic Innovation Challenge. The work from the BLiSS Team, at the University of Michigan, resulted in the design of a system persona, Diego, to which an astronaut may quickly build trust with autonomous systems, to alleviate known stressors on mental health expected during long duration space missions. Optimal software to facilitate integration of the system persona into a reference Lunar orbiting Gateway station was defined. Additionally, a Speech to Text (STT) system and a Graphical User Interface (GUI) that could be implemented in future missions was developed on an Internet of Things (IOT) platform. The Voice User Interface (VUI) design for the M2M X-Hab 2020 project leveraged previous technology developed by the BLiSS team to incorporate a voice-based interface into NASA’s Platform for Autonomous Systems (NPAS) software. This required technologies to convert voice to text, conduct semantic interpretations, and convert responses from the autonomous system to text and to speech; additionally, the spacecraft background noise environment was assessed, a noise mitigation technique was developed, and a relatable personality for the autonomous system was developed in order to facilitate human-like conversations. The success of our effort was largely due to the diversity of the team that included expertise in Space Systems Engineering, Human Computer Interaction, Aerospace Engineering, Computer Science, Biomedical Engineering, and Applied Physics. The diverse perspectives fostered elaborate discussions, resulting in the conception of three main subsystems: (1) User-System, (2) NPAS-System, and (3) Environment-System. The VUI was unique and had to be efficient and intuitive. For this project, 5 subteams were formed, each with a separate objective, Voice Design team, Background Noise Mitigation team, Software Integration team and Graphical User Interface team. The BLiSS team crafted a personality for the VUI to enable human-like conversation and drive user adoption and trust. User surveys were completed and used to help determine the required VUI system personality traits by capturing perspectives and expectations of prospective “Artemis Generation Astronauts”. To further simulate human-like conversations, the system had to be able to quickly interpret user speech and be able to integrate with NASA’s NPAS platform for quick and reliable information transfer. The outcomes of our research were: (1) a working prototype user interface, that is compatible with NASA’s NPAS platform; (2) software that demonstrates the ability of the VUI system to interpret user requests and respond appropriately; (3) the capability to implement fully expanded conversations between user and system using intuitive communication in four request categories; and (4) software and hardware recommendations that optimize the system’s ability to operate in a noisy environment. Our research has laid the foundation for the development of VUI’s for autonomy, and provides a baseline for future VUI developments.

Voice user interface↗
DOE OSTI2017

User-Centered Evaluation of Visual Analytics

Visual analytics systems are becoming very popular. More domains now use interactive visualizations to analyze the ever-increasing amount and heterogeneity of data. More novel visualizations are being developed for more tasks and users. We need to ensure that these systems can be evaluated to determine that they are both useful and usable. A user-centered evaluation for visual analytics needs to be developed for these systems. While many of the typical human-computer interaction (HCI) evaluation methodologies can be applied as is, others will need modification. Additionally, new functionality in visual analytics systems needs new evaluation methodologies. There is a difference between usability evaluations and user-centered evaluations. Usability looks at the efficiency, effectiveness, and user satisfaction of users carrying out tasks with software applications. User-centered evaluation looks more specifically at the utility provided to the users by the software. This is reflected in the evaluations done and in the metrics used. In the visual analytics domain this is very challenging as users are most likely experts in a particular domain, the tasks they do are often not well defined, the software they use needs to support large amounts of different kinds of data, and often the tasks last for months. These difficulties are discussed more in the section on User-centered Evaluation. Our goal is to provide a discussion of user-centered evaluation practices for visual analytics, including existing practices that can be carried out and new methodologies and metrics that need to be developed and agreed upon by the visual analytics community. The material provided here should be of use for both researchers and practitioners in the field of visual analytics. Researchers and practitioners in HCI and interested in visual analytics will find this information useful as well as a discussion on changes that need to be made to current HCI practices to make them more suitable to visual analytics. A history of analysis and analysis techniques and problems is provided as well as an introduction to user-centered evaluation and various evaluation techniques for readers from different disciplines. The understanding of these techniques is imperative if we wish to support analysis in the visual analytics software we develop. Currently the evaluations that are conducted and published for visual analytics software are very informal and consist mainly of comments from users or potential users. Our goal is to help researchers in visual analytics to conduct more formal user-centered evaluations. While these are time-consuming and expensive to carryout, the outcomes of these studies will have a defining impact on the field of visual analytics and help point the direction for future features and visualizations to incorporate. While many researchers view work in user-centered evaluation as a less-than-exciting area to work, the opposite is true. First of all, the goal is user-centered evaluation is to help visual analytics software developers, researchers, and designers improve their solutions and discover creative ways to better accommodate their users. Working with the users is extremely rewarding as well. While we use the term “users” in almost all situations there are a wide variety of users that all need to be accommodated. Moreover, the domains that use visual analytics are varied and expanding. Just understanding the complexities of a number of these domains is exciting. Researchers are trying out different visualizations and interactions as well. And of course, the size and variety of data are expanding rapidly. User-centered evaluation in this context is rapidly changing. There are no standard processes and metrics and thus those of us working on user-centered evaluation must be creative in our work with both the users and with the researchers and developers.

intelligence analysis↗
DOE OSTI2024

Large language model evaluation for high–performance computing software development

We apply AI-assisted large language model (LLM) capabilities of GPT-3 targeting high-performance computing (HPC) kernels for (i) code generation, and (ii) auto-parallelization of serial code in C ++, Fortran, Python and Julia. Our scope includes the following fundamental numerical kernels: AXPY, GEMV, GEMM, SpMV, Jacobi Stencil, and CG, and language/programming models: (1) C++ (e.g., OpenMP [including offload], OpenACC, Kokkos, SyCL, CUDA, and HIP), (2) Fortran (e.g., OpenMP [including offload] and OpenACC), (3) Python (e.g., numpy, Numba, cuPy, and pyCUDA), and (4) Julia (e.g., Threads, CUDA.jl, AMDGPU.jl, and KernelAbstractions.jl). Kernel implementations are generated using GitHub Copilot capabilities powered by the GPT-based OpenAI Codex available in Visual Studio Code given simple + + prompt variants. To quantify and compare the generated results, we propose a proficiency metric around the initial 10 suggestions given for each prompt. For auto-parallelization, we use ChatGPT interactively giving simple prompts as in a dialogue with another human including simple “prompt engineering” follow ups. Results suggest that correct outputs for C++ correlate with the adoption and maturity of programming models. For example, OpenMP and CUDA score really high, whereas HIP is still lacking. We found that prompts from either a targeted language such as Fortran or the more general-purpose Python can benefit from adding language keywords, while Julia prompts perform acceptably well for its Threads and CUDA.jl programming models. Finally, we expect to provide an initial quantifiable point of reference for code generation in each programming model using a state-of-the-art LLM. Overall, understanding the convergence of LLMs, AI, and HPC is crucial due to its rapidly evolving nature and how it is redefining human-computer interactions.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2015

2015 Space Human Factors Engineering Standing Review Panel

The 2015 Space Human Factors Engineering (SHFE) Standing Review Panel (from here on referred to as the SRP) met for a site visit in Houston, TX on December 2 - 3, 2015. The SRP reviewed the updated research plans for the Risk of Inadequate Design of Human and Automation/Robotic Integration (HARI Risk), the Risk of Inadequate Human-Computer Interaction (HCI Risk), and the Risk of Inadequate Mission, Process and Task Design (MPTask Risk). The SRP also received a status update on the Risk of Incompatible Vehicle/Habitat Design (Hab Risk) and the Risk of Performance Errors Due to Training Deficiencies (Train Risk). The SRP is pleased with the progress and responsiveness of the SHFE team. The presentations were much improved this year. The SRP is also pleased with the human-centered design approach. Below are some of the more extensive comments from the SRP. We have also made comments in each section concerning gaps/tasks in each. The comments below reflect more significant changes that impact more than just one particular section.

Steinberg, Susan↗
NASA NTRS1991

OFMspert: An architecture for an operator's associate that evolves to an intelligent tutor

With the emergence of new technology for both human-computer interaction and knowledge-based systems, a range of opportunities exist which enhance the effectiveness and efficiency of controllers of high-risk engineering systems. The design of an architecture for an operator's associate is described. This associate is a stand-alone model-based system designed to interact with operators of complex dynamic systems, such as airplanes, manned space systems, and satellite ground control systems in ways comparable to that of a human assistant. The operator function model expert system (OFMspert) architecture and the design and empirical validation of OFMspert's understanding component are described. The design and validation of OFMspert's interactive and control components are also described. A description of current work in which OFMspert provides the foundation in the development of an intelligent tutor that evolves to an assistant, as operator expertise evolves from novice to expert, is provided.

Mitchell, Christine M.↗
NASA NTRS2017

The NASA Augmented/Virtual Reality Lab: The State of the Art at KSC

The NASA Augmented Virtual Reality (AVR) Lab at Kennedy Space Center is dedicated to the investigation of Augmented Reality (AR) and Virtual Reality (VR) technologies, with the goal of determining potential uses of these technologies as human-computer interaction (HCI) devices in an aerospace engineering context. Begun in 2012, the AVR Lab has concentrated on commercially available AR and VR devices that are gaining in popularity and use in a number of fields such as gaming, training, and telepresence. We are working with such devices as the Microsoft Kinect, the Oculus Rift, the Leap Motion, the HTC Vive, motion capture systems, and the Microsoft Hololens. The focus of our work has been on human interaction with the virtual environment, which in turn acts as a communications bridge to remote physical devices and environments which the operator cannot or should not control or experience directly. Particularly in reference to dealing with spacecraft and the oftentimes hazardous environments they inhabit, it is our hope that AR and VR technologies can be utilized to increase human safety and mission success by physically removing humans from those hazardous environments while virtually putting them right in the middle of those environments.

Little, William↗
NASA NTRS1993

Intelligent systems and advanced user interfaces for design, operation, and maintenance of command management systems

Historically, command management systems (CMS) have been large and expensive spacecraft-specific software systems that were costly to build, operate, and maintain. Current and emerging hardware, software, and user interface technologies may offer an opportunity to facilitate the initial formulation and design of a spacecraft-specific CMS as well as to develop a more generic CMS system. New technologies, in addition to a core CMS common to a range of spacecraft, may facilitate the training and enhance the efficiency of CMS operations. Current mission operations center (MOC) hardware and software include Unix workstations, the C/C++ programming languages, and an X window interface. This configuration provides the power and flexibility to support sophisticated and intelligent user interfaces that exploit state-of-the-art technologies in human-machine interaction, artificial intelligence, and software engineering. One of the goals of this research is to explore the extent to which technologies developed in the research laboratory can be productively applied in a complex system such as spacecraft command management. Initial examination of some of these issues in CMS design and operation suggests that application of technologies such as intelligent planning, case-based reasoning, human-machine systems design and analysis tools (e.g., operator and designer models), and human-computer interaction tools (e.g., graphics, visualization, and animation) may provide significant savings in the design, operation, and maintenance of the CMS for a specific spacecraft as well as continuity for CMS design and development across spacecraft. The first six months of this research saw a broad investigation by Georgia Tech researchers into the function, design, and operation of current and planned command management systems at Goddard Space Flight Center. As the first step, the researchers attempted to understand the current and anticipated horizons of command management systems at Goddard. Preliminary results are given on CMS commonalities and causes of low re-use, and methods are proposed to facilitate increased re-use.

Potter, William J.↗