Search NASA⌕ Search

SEARCH · Search NASA

Results for “Voice interface for autonomous systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

8 records
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Enabling a Voice Management System for Space Applications

The sustainable missions beyond Low Earth Orbit (LEO) envisioned for NASA’s Artemis program will require autonomous capabilities. Moreover, Artemis mission crews will need a means to efficiently interact with a spacecraft’s autonomous systems. This interaction can be facilitated by voice and speech communications because voice-based controls enable users to interact hands- and eyes-free, allowing the user to better focus on critical tasks. The goal of our project was to explore the knowledge and technology needed to successfully design effective Voice User Interfaces (VUIs) for autonomous systems utilizing Human Centered Design (HCD) principles. The focus of the human factors’ aspect of engineering, pays close attention to psychological and physiological principles in the development of autonomous crew operation systems. A main objective was to understand how a crew member, through voice interaction, could efficiently and intuitively communicate with a notional autonomous vehicle system manager. This project was a part of the NASA Moon to Mars eXploration Systems and Habitation (M2M X-Hab) 2020 Academic Innovation Challenge. The work from the BLiSS Team, at the University of Michigan, resulted in the design of a system persona, Diego, to which an astronaut may quickly build trust with autonomous systems, to alleviate known stressors on mental health expected during long duration space missions. Optimal software to facilitate integration of the system persona into a reference Lunar orbiting Gateway station was defined. Additionally, a Speech to Text (STT) system and a Graphical User Interface (GUI) that could be implemented in future missions was developed on an Internet of Things (IOT) platform. The Voice User Interface (VUI) design for the M2M X-Hab 2020 project leveraged previous technology developed by the BLiSS team to incorporate a voice-based interface into NASA’s Platform for Autonomous Systems (NPAS) software. This required technologies to convert voice to text, conduct semantic interpretations, and convert responses from the autonomous system to text and to speech; additionally, the spacecraft background noise environment was assessed, a noise mitigation technique was developed, and a relatable personality for the autonomous system was developed in order to facilitate human-like conversations. The success of our effort was largely due to the diversity of the team that included expertise in Space Systems Engineering, Human Computer Interaction, Aerospace Engineering, Computer Science, Biomedical Engineering, and Applied Physics. The diverse perspectives fostered elaborate discussions, resulting in the conception of three main subsystems: (1) User-System, (2) NPAS-System, and (3) Environment-System. The VUI was unique and had to be efficient and intuitive. For this project, 5 subteams were formed, each with a separate objective, Voice Design team, Background Noise Mitigation team, Software Integration team and Graphical User Interface team. The BLiSS team crafted a personality for the VUI to enable human-like conversation and drive user adoption and trust. User surveys were completed and used to help determine the required VUI system personality traits by capturing perspectives and expectations of prospective “Artemis Generation Astronauts”. To further simulate human-like conversations, the system had to be able to quickly interpret user speech and be able to integrate with NASA’s NPAS platform for quick and reliable information transfer. The outcomes of our research were: (1) a working prototype user interface, that is compatible with NASA’s NPAS platform; (2) software that demonstrates the ability of the VUI system to interpret user requests and respond appropriately; (3) the capability to implement fully expanded conversations between user and system using intuitive communication in four request categories; and (4) software and hardware recommendations that optimize the system’s ability to operate in a noisy environment. Our research has laid the foundation for the development of VUI’s for autonomy, and provides a baseline for future VUI developments.

Voice user interface↗
NASA NTRSDate not supplied

Enabling a Voice Management System for Space Applications, Design and Software Development

Sustainable missions, beyond low Earth orbit, will require autonomous capabilities in order to achieve NASA’s Artemis program objectives. Correspondingly, the crew must have a means to efficiently interact with these autonomous systems; this can be facilitated via voice and speech communications. Voice-based controls enable the user to access autonomous systems hands-free/eyes-free, allowing the user to better focus on critical tasks. The goal of this project was to explore the knowledge and technology needed to successfully design effective voice interfaces for autonomous systems. The main objective was to understand how a crew member, through voice interaction, could most efficiently and intuitively communicate with a notional autonomous vehicle system manager. This project leveraged prior research conducted by the University of Michigan’s Bioastronautics and Life Support System (BLiSS) team as part of a NASA Moon to Mars eXploration Systems and Habitation (M2M X-Hab) 2020 Academic Innovation Challenge. The X-Hab 2020 work from the BliSS Team resulted in an intuitive graphical user interface/user experience that was built on an Internet of Things (IOT) platform. The Voice User Interface (VUI) design for the M2M X-Hab 2021 project leveraged this technology and incorporated a voice-based assistant and NASA’s Platform for Autonomous Systems (NPAS) software. This required technologies to convert voice to text, conduct semantic interpretations, and convert responses from the autonomous system to text and to speech; additionally, the background noise environment of spacecraft was assessed, and a relatable personality for the autonomous system to facilitate human-like conversations was created. This work’s success was largely due to the diverse team that included expertise in Space Systems Engineering, Human Computer Interaction, Aerospace Engineering, Computer Science, Biomedical Engineering, and Applied Physics. The differing perspectives fostered elaborate discussions, resulting in the conception of three main interactions: (1) User-System, (2) NPAS-System, and (3) Environment-System. The system developed, i.e. the VUI, had to be unique, efficient, and intuitive; thus, the team crafted a personality for the system to enable human-like conversation. User surveys sent to students and young professionals were used to help determine these personality traits by capturing perspectives and expectations of the “Artemis Generation Astronauts”. To further simulate human-like conversations, the system had to be able to quickly interpret user speech and be able to integrate with NASA’s NPAS system for quick and reliable information transfer. Results of this research include (1) a working prototype user interface, that is compatible with NASA’s NPAS system; (2) software that demonstrates the ability to interpret user requests and respond appropriately; (3) the capability to implement fully expanded conversations between user and system using intuitive communication in four request categories; and (4) software and hardware recommendations that optimize the system’s ability to operate, i.e. be heard, in a noisy environment. The technologies chosen for this project’s demonstrations included the following: Raspberry Pi, RASA, Mozilla Deep Speech, Coqui, RTX Voice and Adobe XD. This work has laid the foundation for the development of VUI’s used for autonomy, and is intended to provide guidance for future VUI development.

Tara Vega↗
NASA NTRS2008

Architecting Communication Network of Networks for Space System of Systems

The National Aeronautics and Space Administration (NASA) and the Department of Defense (DoD) are planning Space System of Systems (SoS) to address the new challenges of space exploration, defense, communications, navigation, Earth observation, and science. In addition, these complex systems must provide interoperability, enhanced reliability, common interfaces, dynamic operations, and autonomy in system management. Both NASA and the DoD have chosen to meet the new demands with high data rate communication systems and space Internet technologies that bring Internet Protocols (IP), routers, servers, software, and interfaces to space networks to enable as much autonomous operation of those networks as possible. These technologies reduce the cost of operations and, with higher bandwidths, support the expected voice, video, and data needed to coordinate activities at each stage of an exploration mission. In this paper, we discuss, in a generic fashion, how the architectural approaches and processes are being developed and used for defining a hypothetical communication and navigation networks infrastructure to support lunar exploration. Examples are given of the products generated by the architecture development process.

Bhasin, Kul B.↗
NASA NTRSDate not supplied

Preliminary Design of an 'Autonomous Medical Response Agent' Interface Prototype for Long Duration Spaceflight

Major challenges for astronauts in future long-duration exploration missions (LDEMs) will be that crewmembers are not expected to be medical professionals, may be under high workload and stress, are facing physiological challenges caused by spaceflight, and will have limited, delayed voice communications with medical support from Earth. An autonomous medical response agent (AMRA) is envisioned to help astronauts address medical complaints, develop a differential diagnosis, and guide self-treatment until a healthy state is restored. AMRA develops a process of personalized diagnosis and treatment through a Bayesian predictive control system that recommends therapeutic control actions including diagnostic tests and treatments to crewmembers (Menon, 2020). The Human Computer Interaction (HCI) lab from NASA Ames Research Center’s Human Systems Integration Division (Code TH) has collaborated with Nahlia Inc in human-centered design augmentation research for AMRA. The project, titled Design of ‘Autonomous Medical Response Agent Interface Prototype for Long Duration Spaceflight, has been funded by the Translational Research Institute for Space Health (TRISH) and introduces an interactive user-interface prototype that guides astronauts through self-diagnosis, treatment, and rehabilitation while communicating with remote specialists in ground support (most notably a patient’s flight surgeon). Our project develops the interaction design for the crewmember using AMRA through user research, iterative design, and usability testing to evaluate the user interface and workflow designed. The interface design deliverable for this project, titled AMRA Aggregate Information Display (AMRA AID) is an integrated information display system for comprehensive autonomous medical guidance, diagnosis, and treatment of in-flight medical conditions experienced by crewmembers. AMRA AID demonstrates how we might ensure crew autonomy, increase the crew’s medical capabilities, and decrease cognitive burden within a front-end user interface. AMRA AID refrains from relying on input from ground or mission control for self-treatment of medical issues—though ground awareness and communication with ground is maintained as a means of ensuring trust between mission control and crew. AMRA AID demonstrates how the crew’s on-board medical system might integrate with information from vehicle monitoring and crew schedule, without assuming causal relationships. AMRA AID’s comprehensive view enables efficient information access for both crew and ground support, reducing cognitive burden in the event of an unplanned or emergency medical incident and enabling informed analytical decisions to be made based on both crew and vehicle health. Human-centered design augmentation advanced within the prototype included: enhanced workflow and treatment guidance for two medical scenarios for a non-specialist user base with various levels of medical training, interaction design which considered speech (conversational user interface) elements and on-screen interactions to be developed in future iterations of the project, communication design and functional requirements relevant to self-care versus caring for another astronaut, as well as user testing of the prototype with an international space medical community. This project arrives at critical findings regarding usability needs, communication requirements, and integrated information requirements for a future technology interface functioning to increase confidence between ground support and LDEM crewmembers.

TRISH↗
NASA NTRSDate not supplied

Autonomy Voice Assistant for NPAS (NASA Platform for Autonomous Systems)

A prototype voice interaction system, Autonomy Voice Assistant (AVA), is described in this paper. AVA is designed to seamlessly integrate into the NASA Platform for Autonomous Systems (NPAS), an autonomy software platform, and to enable an operator to interact with NPAS autonomy applications through voice conversations. By integrating VA with NPAS, a major enhancement to NPAS applications is facilitated, enabling interaction through natural language expressions. An AVA prototype has been designed incorporating two principles:(1) self-containment (no external data or computations required), and (2) a readily modifiable, reconfigurable, and flexible architecture. By using voice messages in an NPAS application, an additional layer of user interface capability is enabled, thereby enhancing a user’s overall experience. Advancements, over the past several decades in speech recognition and natural language processing technologies has made it possible for AVA to implement robust messaging capabilities while still being lightweight. The main objective of incorporating a voice assistant like AVA is to augment the number and effectiveness of interactions a user has with a system that typically uses mouse-based interaction, while simultaneously enriching the user experience and providing heightened system awareness.

Lucian Murdock↗
NASA NTRS1988

Distributed cooperating processes in a mobile robot control system

A mobile inspection robot has been proposed for the NASA Space Station. It will be a free flying autonomous vehicle that will leave a berthing unit to accomplish a variety of inspection tasks around the Space Station, and then return to its berth to recharge, refuel, and transfer information. The Flying Eye robot will receive voice communication to change its attitude, move at a constant velocity, and move to a predefined location along a self generated path. This mobile robot control system requires integration of traditional command and control techniques with a number of AI technologies. Speech recognition, natural language understanding, task and path planning, sensory abstraction and pattern recognition are all required for successful implementation. The interface between the traditional numeric control techniques and the symbolic processing to the AI technologies must be developed, and a distributed computing approach will be needed to meet the real time computing requirements. To study the integration of the elements of this project, a novel mobile robot control architecture and simulation based on the blackboard architecture was developed. The control system operation and structure is discussed.

Skillman, Thomas L., Jr.↗
NASA NTRS2023

Safety Case for Small Uncrewed Aircraft Systems (sUAS) Beyond Visual Line of Sight (BVLOS) Operations at NASA Langley Research Center

This Technical Memorandum (TM) is written to provide for dissemination of the methods and safety considerations for operations of small Uncrewed Aerial Systems (sUAS) Beyond Visual Line-of-Sight (BVLOS)at NASA Langley Research Center. It includes the Safety Case used to acquire a BVLOS Certificate of Authorization (COA) from the FAA and is being published to enable others to benefit from this work. The intended operations, subject to approval from the Federal Aviation Administration (FAA) and the National Aeronautics and Space Administration (NASA), will include a combination of Within Visual Line of Sight (WVLOS) and Beyond Visual Line of Sight (BVLOS) flights, comprising of at most five sUAS operating concurrently, with no more than three operating BVLOS. Flights will occur in a subset of the Langley Air Force Base (LAFB) Class D airspace (KLFI) at a maximum altitude of 400 ft AGL. Most operations within this subset will take place in the City Environment Range Testing for Autonomous Integrated Navigation (CERTAIN) Range. The CERTAIN Range includes airspace inside the borders of NASA Langley Research Center (LaRC). Additional airspace over the northern section of CERTAIN will be requested as part of the Certificate of Authorization (COA). NASA LaRC BVLOS operations on the CERTAIN Range can be broken down into five critical components needed to meet the 14 CFR § 91.113 see and avoid requirement: 1) procedural deconfliction with LAFB for UAS operations at or below 400 ft and manned aircraft at or above 900’ AGL; 2) ground equipment for detection of intruder aircraft and to support communications between crewmembers ; 3) sUAS vehicles with advanced onboard automation capable of autonomously maintaining safe separation; 4) BVLOS standardized operating procedures (SOPs); 5) and personnel to execute the flight operations in accordance with the SOPs and respond to airborne contingencies. The introduction of new ground equipment includes the use of the Remote Operations for Autonomous Missions (ROAM) UAS Operations Center, development and use of an Integrated Airspace Display (IAD), use of the L-STAR and GA-9120 radars, and the incorporation of standardized Vertiports. The ROAM Operations Center will be the central point for all BVLOS sUAS operations. All command and control (C2), voice communications and airspace awareness displays will reside inside ROAM. The IAD will provide raw data from ADS-B, FLARM, radar tracks and telemetered GPS vehicle positions for interpretation by an Airspace Monitor. The radars will search the class D airspace around the CERTAIN Range and serve as a backup to procedural deconfliction procedures coordinated with LAFB. In the event of a procedural deconfliction breakdown, radar detections of non-participating aircraft will be available so that the 91.113 see and avoid requirement can still be safely met. Finally, the incorporation of Vertiports will have video and network connectivity that enables large numbers of sUAS launches and recoveries from a single location. This is a continuation of the remote command and control of unpiloted aircraft component focused on evaluating unpiloted aircraft flight crew roles and responsibilities, control interfaces and the associated data links needed to operate a fleet of aircraft within a UAM Ecosystem. This work supports the development of future aviation operational concepts based on an Urban Air Mobility Maturity Level (UML) 4 environment (Patterson, 2020). It is assumed that future airspace will include hundreds of simultaneous aircraft operations within the airspace, therefore scalable operations are essential for enabling this future airspace to become a reality. Follow on work includes envisioned flights that expand operations beyond the CERTAIN range and lead to an effective Maritime Surveillance capability.

Matthew W Coldsnow↗
NASA NTRSDate not supplied

A Notional Artemis Lunar Surface Exploration Package (ArLSEP) based on the Gandalf Staff Platform

Introduction: The Artemis program is planning to deliver crew and cargo to the lunar surface, but there is no current package for supporting lunar in-struments and experiments similar to the Apollo Lunar Surface Exploration Package (ALSEP). This abstract provides a possible concept for such a package using the Gandalf Staff Platform as a common core. Gandalf Staff: The Gandalf Staff is an early prototype system developed over FY’21/FY’22 using NASA Science Technology Mission Directorate (STMD) Center Information Fund (CIF) grants to de-sign, build and test “proof-of-concept” components. These components include a 24v battery powered monopole that powers a suite of subsystems, including a Graphical User Interface (GUI) for crew, surface voice and data communications, Lunar Search and Rescue (LunaSAR) navigation and communications, LiDAR, field site external lighting, 360-degree camera, and a geothermal instrument for measuring sub-surface temperature gradient. The staff can be carried independently by an Extra-Vehicular Activity (EVA) astronaut, or can be mounted into a tripod for “hands free” support at a surface site being investigated. The staff can be attached to an external solar array and power storage system for long-duration operations. [1,2] ALSEP: An ASLEP flew on each mission Apollo 12 to Apollo 17. For Apollo 11, a simplified packaged called the Early Apollo Scientific Experiments Pack-age (EASEP) was flown. Each package included a “Central Station” that provided the power and communications connected to a variety of instruments and sensors. The power was provided by a Radioisotope Thermoelectric Generator (RTG) fueled by Plutoni-um-238 generating 70 watts of power (initially, decayed over time) [3]. The communications system provide for direct to Earth data transfer from the lunar surface. Each pack-age was stowed externally in the Lunar Module (LM) Scientific Equipment (SEQ) bay with a mass up to 163 kg (Apollo 17). The crew unloaded the ALSEP from the LM and deployed the instruments on the lunar surface. Although designed to operate for only 1 year, many sites operated for up to 8 years successfully [4]. The Active Seismic Experiment (ASE) included 3 geophones for detecting seismic waves created by mortars and thumpers deployed by the crew. Other active experiments measured the lunar atmosphere, the heat flow in the subsurface, the lunar gravity and potential gravity waves, the lunar magnetic field, the solar wind and plasma interactions in cislunar space. Passive experiments included collectors for dust and cosmic rays, and retroreflectors for precise measurements of distance using a laser from Earth. The ALSEP program continues to generate insights into lunar formation and evolution. ArLSEP Concepts: The lunar surface science package for the Artemis program will hopefully exceed the capability of the ALSEP. There are multiple issues for discussion leading to the design of a new ArLSEP, needing requirements definition from the science community, NASA mission architecture, and NASA budget planners. 1. Delivery Mechanism Two possible projects currently provide capability to deliver scientific cargo to the lunar surface: 1) the Commercial Lunar Payload Services (CLPS) [5] and the Human Landing System (HLS) [6, 7]. Each project is controlled by a different organization within NASA and budgeted with different criteria although both support lunar exploration. The HLS system delivers crew (and potentially cargo) to human landing sites. If an ArLSEP is “predeployed” to such a site, the design must include power (either from the vehicle or independently) to keep the electronics functioning until deployed by the crew. If an ArLSEP is delivered on a vehicle after the crew is present on the lunar surface, safety protocols require adequate distance from the humans for impact from descent propelled sur-face regolith ejecta. This distance can not exceed the capability of the crew to walk (if no rover) to the vehicle for ArLSEP deployment. 2. Overall Guidelines The general design of ArLSEP will likely follow the ALSEP with a common system for communications and power; however, significant architecture differences between Apollo and Artemis exist. Power: The RTG will not be available for early Artemis missions nor likely follow-on Lunar Exploration Transportation Services (LETS) missions [8]. Thus, ArLSEP power must be supplied by solar arrays with sufficient battery capability to “keep alive” necessary electronics during any lunar surface eclipse period. Communication: The Artemis program is developing a series of communications satellites for lunar orbit to provide surface transmission of data and voice to Earth. Called “LunaNET”, this network is component useful for ArLSEP since south polar locations may not always have direct “line-of-sight” to Earth [9]. 3. Concept of Operations (ConOps) The general ConOps for ArLSEP is to deliver the package to lunar surface before the crew arrives, and then have the crew deploy the package after some period of time. This requires coordinated design (for power systems) and launch window (for schedule) on both the cargo and crew missions. Once the ArLSEP is deployed, it will operate autonomously for a number of years. It should be designed to be EVA compatible for crew maintenance and upgrade. 4. Notional Design (for discussion purpose only) The landing site near the South Pole is expected to have no eclipse cycle exceeding 5 days, so the “keep alive” power is 144 hours (6 days to include margin). A 12v ArLSEP will use rechargeable LiFePO4 cells, which are common in the Electric Vehicle (EV) industry. With a current of 5 amps and a 125 watt system, the mass is about 90kg. The comm. system and structure adds another 10kg, thus the “Central Station” is approximately 100kg. The solar power is collected on four arrays (each 2m above the surface), and the entire ArLSEP is designed to stow in a 2m x 1m x 1m volume. The experiment and instrument design will vary for each installation and add mass to the total (although they are expected to fit within the 2m3 volume). Seismic wave generation will likely not be provided with mortars, thus an electric “thumper” will be required. Active instruments such as imaging systems and sensing instruments will benefit from the additional power and communication capability provided by ArLSEP. Passive systems such as retroreflectors, witness plates, and cosmic dust collectors can be added to either the landing vehicle and/or the ArLSEP. With repeated HLS missions to the same human site, the ArLSEP can be expanded and easily maintained for long duration science collection on the lunar surface.

ALSEP↗