Augmented Reality intelligent Crew Assistant
Explore the source record for details and available documents.
Augmented Reality↗
SEARCH · Search NASA
Results for “Intelligent Crew Assistant”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Deep Learning for Space Applications
Explore the source record for details and available documents.
Augmented Reality↗
Enabling Autonomous On-Board Trouble-Shooting
As NASA missions reach deeper into space, a few crew members will need to do the work of 10s-100s of Ground Controllers, most significantly, in critical areas such as trouble-shooting anomalies that could result in loss of crew or loss of mission. Understanding what the risk is of these sorts of events based on the history of the International Space Station allows us to anticipate how critical this challenge will be for future crewed missions, and allowing us to define the research that must be carried out in order to establish Standards, Guidelines and Requirements for on-board intelligent technologies that can assist crew as needed.
autonomy↗
Crew Time Requirements in Future Space Greenhouses - What Can We Infer from Current Analog and Space Missions?
Efficient crop production will be required to advance humanity’s presence in space, and for this, accurate predictions of crew time in future space greenhouse modules will be crucial to design and operate these plant growth systems, and schedule crop production. Crew time estimates will also be critical for deciding priorities of automating different aspects of space crop production. Because it is difficult to capture in operational environments, crew time for plant cultivation has only been sporadically recorded in past analog and space missions. We propose a methodology for efficient categorizing and reporting of crew time in space plant growth systems: first identify the different tasks needed to operate the greenhouse module, second define a representative time period for data collection, third accurately report crew time for individual tasks - and their occurrence, and fourth use collected data to improve greenhouse modules and plant growth system designs. Using data from various analog facilities and from the Veggie hardware on ISS, and assumptions for different mission scenarios, we discuss how crew time for plant cultivation can be reduced with adequate choices of crops, automation, artificial intelligence (AI) and virtual assistants, and sufficient crew training. This has major implications for the design of future space greenhouse modules. For example, missions on future space stations or during interplanetary travel would save significant crew time by including leafy greens and microgreens for astronaut’s diet supplement, with automated watering, health and environmental checks, as well as AI managing maintenance schedules, and a virtual assistant for repair activities. This work was funded by NASA Space Biology through NASA postdoctoral program / USRA, by NASA’s Space Biology and Human Research Programs, and by the European Union Horizon 2020 program via the COMPET-07-2014 - Space exploration – Life-support subprogram (reference number: 636501).
Lucie Poulet↗
Crew Time Requirements in Future Space Greenhouses: What Can We Infer from Current Analog and Space Missions?
Efficient crop production will be required to advance humanity’s presence in space, and for this, accurate predictions of crew time in future space greenhouse modules will be crucial to design and operate these plant growth systems, and schedule crop production. Crew time estimates will also be critical for deciding priorities of automating different aspects of space crop production. Because it is difficult to capture in operational environments, crew time for plant cultivation has only been sporadically recorded in past analog and space missions. We propose a methodology for efficient categorizing and reporting of crew time in space plant growth systems: first identify the different tasks needed to operate the greenhouse module, second define a representative time period for data collection, third accurately report crew time for individual tasks - and their occurrence, and fourth use collected data to improve greenhouse modules and plant growth system designs. Using data from various analog facilities and from the Veggie hardware on ISS, and assumptions for different mission scenarios, we discuss how crew time for plant cultivation can be reduced with adequate choices of crops, automation, artificial intelligence (AI) and virtual assistants, and sufficient crew training. This has major implications for the design of future space greenhouse modules. For example, missions on future space stations or during interplanetary travel would save significant crew time by including leafy greens and microgreens for astronaut’s diet supplement, with automated watering, health and environmental checks, as well as AI managing maintenance schedules, and a virtual assistant for repair activities. This work was funded by NASA Space Biology through NASA postdoctoral program / USRA, by NASA’s Space Biology and Human Research Programs, and by the European Union Horizon 2020 program via the COMPET-07-2014 - Space exploration – Life-support subprogram (reference number: 636501).
Veggie↗
Enhancing Situation Awareness of Automated Procedures Using Adaptive Multimodal Augmented Reality Displays
In future missions, crew must be more autonomous and perform tasks with limited help from MCC. Virtual Intelligent Task Assistant (VITA) will stand-in for MCC and help crew perform tasks. Prepare for and perform manual tasks to increase efficiency and accuracy. Take over partially completed task by summarizing progress made so far, and reviewing remaining tasks. Maintain situation awareness of ongoing automated procedures while crew is performing manual tasks.
Schreckenghost, D.↗
Autonomous Agents and Intelligent Assistants for Exploration Operations
Human exploration of space will involve remote autonomous crew and systems in long missions. Data to earth will be delayed and limited. Earth control centers will not receive continuous real-time telemetry data, and there will be communication round trips of up to one hour. There will be reduced human monitoring on the planet and earth. When crews are present on the planet, they will be occupied with other activities, and system management will be a low priority task. Earth control centers will use multi-tasking "night shift" and on-call specialists. A new project at Johnson Space Center is developing software to support teamwork between distributed human and software agents in future interplanetary work environments. The Engineering and Mission Operations Directorates at Johnson Space Center (JSC) are combining laboratories and expertise to carry out this project, by establishing a testbed for hWl1an centered design, development and evaluation of intelligent autonomous and assistant systems. Intelligent autonomous systems for managing systems on planetary bases will commuicate their knowledge to support distributed multi-agent mixed-initiative operations. Intelligent assistant agents will respond to events by developing briefings and responses according to instructions from human agents on earth and in space.
Malin, Jane T.↗
Operator modeling in commerical aviation: Cognitive models, intelligent displays, and pilot's assistants
One of the goals of the National Aviation Safety/Automation program is to address the issue of human-centered automation in the cockpit. Human-centered automation is automation that, in the cockpit, enhances or assists the crew rather than replacing them. The Georgia Tech research program focused on this general theme, with emphasis on designing a computer-based pilot's assistant, intelligent (i.e, context-sensitive) displays, and an intelligent tutoring system for understanding and operating the autoflight system. In particular, the aids and displays were designed to enhance the crew's situational awareness of the current state of the automated flight systems and to assist the crew's situational awareness of the current state of the automated flight systems and to assist the crew in coordinating the autoflight system resources. The activities of this grant included: (1) an OFMspert to understand pilot navigation activities in a 727 class aircraft; (2) an extension of OFMspert to understand mode control in a glass cockpit, Georgia Tech Crew Activity Tracking System (GT-CATS); (3) the design of a training system to teach pilots about the vertical navigation portion of the flight management system -VNAV Tutor; and (4) a proof-of-concept display, using existing display technology, to facilitate mode awareness, particularly in situations in which controlled flight into terrain (CFIT) is a potential.
Govindaraj, T.↗
An evaluation of a real-time fault diagnosis expert system for aircraft applications
A fault monitoring and diagnosis expert system called Faultfinder was conceived and developed to detect and diagnose in-flight failures in an aircraft. Faultfinder is an automated intelligent aid whose purpose is to assist the flight crew in fault monitoring, fault diagnosis, and recovery planning. The present implementation of this concept performs monitoring and diagnosis for a generic aircraft's propulsion and hydraulic subsystems. This implementation is capable of detecting and diagnosing failures of known and unknown (i.e., unforseeable) type in a real-time environment. Faultfinder uses both rule-based and model-based reasoning strategies which operate on causal, temporal, and qualitative information. A preliminary evaluation is made of the diagnostic concepts implemented in Faultfinder. The evaluation used actual aircraft accident and incident cases which were simulated to assess the effectiveness of Faultfinder in detecting and diagnosing failures. Results of this evaluation, together with the description of the current Faultfinder implementation, are presented.
Schutte, Paul C.↗
Advanced flight experiment automation development
NASA's space science program suffers under operational constraints that severely limit scientific reactivity. We describe a possible extension to an existing system that enables space science to be conducted in a more reactive manner through advanced automation techniques that have recently been used in the Space Transportation System (STS)-based Spacelab. This automation allows sophisticated autonomous control of the experiment by monitoring and dynamically controlling the progress of the experiment protocol. This new approach permits intelligent conduct of experimental research investigations whether or not the crew is available or contact with the ground is possible. We describe an intelligent software assistant that can manage the conduct of the experiment autonomously, by providing responsive telepresence science, or by assisting the astronaut operator with scientific advice. The crew could be unavailable for short periods when the experiment has low priority, or for longer periods on partially tended Space Station. We suggest this system could be developed as a ground prototype using commercial hardware, versions of which have been flight qualified, in order to provide a natural transition path towards flight use. In order to make this demonstration most valuable, we propose not to select a particular Life Sciences experiment, but to include an experiment simulator representative of major scientific domains.
Groleau, Nicolas↗
Limitations and Feasibility of Mini X-Ray Devices in Space Environments
LIMITATIONS AND FEASIBILITY OF MINI X-RAY DEVICES IN SPACE ENVIRONMENTS As space exploration advances toward long-duration missions, reliable medical diagnostic tools become increasingly critical. The miniature x-ray (XR) technology demonstrations by the Exploration Medical Capability (ExMC) and the Exploration Medical Integrated Product Team (XMIPT) aim to assess the feasibility and utility of miniature XR devices in spaceflight. This abstract explores the limitations of current miniature XR systems, the challenges of training crew members, the potential role of clinical decision support systems (CDSS), and the feasibility of ground-based image interpretation. We also propose the integration of miniature XR into other ExMC efforts aimed at identifying the capabilities and resources needed for future exploration class missions. One of the primary challenges with miniature XR devices is the ability to achieve specific anatomical views, particularly in the confined and weightless conditions of a spacecraft. Operators may struggle to acquire diagnostic-quality images when space is limited for proper patient positioning and the volume of the imaging device. Since space radiation and detector limitations may further impact image quality, the flexibility of the operating procedures of these devices will be critical for their success in space applications. CHALLENGES IN TRAINING CREW TO OPERATE IMAGING DEVICES Training in the skills necessary to acquire diagnostic-quality scans may be a barrier for non-clinician crewmembers. The curriculum developed for crew medical officers (CMOs) will require simplification and adaptation to fit into the highly truncated pre-flight training period. Therefore, hands-on familiarization and simulation, both pre-flight and just-in-time training during missions, will be crucial to ensuring the crew can operate the devices in real-life situations. The ability to adjust acquisition parameters must be simplified or made automatic through exam selections on equipment user interfaces, and subject and operator positioning should be assisted with laser guidance and pictorial guides. POTENTIAL FOR CDSS OR ARTIFICIAL INTELLIGENCE (AI)-ASSISTED CDSS CDSS and AI-assisted CDSS offer significant promise in assisting crew members with limited medical training. These systems could provide real-time feedback on image quality and interpretation, helping to mitigate the risks of human error during space missions. Integrating procedural guidance tools, such as virtual and augmented reality, will support crewmembers in accurately positioning patients and obtaining high-quality images. However, the success of such systems will depend on the development of robust training datasets, integration with spaceflight-rated hardware, and the medical decision-making capabilities of operators. FEASIBILITY OF GROUND INTERPRETATION AND DATA TRANSMISSION Reliance on ground-based interpretation may prove difficult for acute care during exploration class-missions due to delays in transmission with increasing distance from Earth or complete communication blackout periods. In such instances where immediate interpretation for clinical intervention is required, crew must be able to interpret the images independently or utilize AI-based assistance to do so. File sizes for XR exams can also be large if numerous images are acquired and bandwidth constraints may limit data transmissions for both radiography and ultrasound exams. FUTURE WORK AND INTEGRATION INTO THE EVIDENCE LIBRARY Future work proposes integrating miniature XR devices into NASA’s Evidence Library to address medical conditions identified as significant contributors to crew morbidity and mortality. The possibility of combining miniature XR with other imaging modalities, such as ultrasound devices, is also under investigation. In conclusion, while miniature XR technology holds potential for extraterrestrial medical systems, there are significant challenges to overcome. Training, integration of AI tools, dedicated exam protocols for microgravity, and improved data transmission systems will be key to realizing the full benefits of miniature XR technology in space.
A M Nelson↗
Operational efficiency subpanel advanced mission control
Herein, the term mission control will be taken quite broadly to include both ground and space based operations as well as the information infrastructure necessary to support such operations. Three major technology areas related to advanced mission control are examined: (1) Intelligent Assistance for Ground-Based Mission Controllers and Space-Based Crews; (2) Autonomous Onboard Monitoring, Control and Fault Detection Isolation and Reconfiguration; and (3) Dynamic Corporate Memory Acquired, Maintained, and Utilized During the Entire Vehicle Life Cycle. The current state of the art space operations are surveyed both within NASA and externally for each of the three technology areas and major objectives are discussed from a user point of view for technology development. Ongoing NASA and other governmental programs are described. An analysis of major research issues and current holes in the program are provided. Several recommendations are presented for enhancing the technology development and insertion process to create advanced mission control environments.
Friedland, Peter↗
Design of an intelligent information system for in-flight emergency assistance
The present research has as its goal the development of AI tools to help flight crews cope with in-flight malfunctions. The relevant tasks in such situations include diagnosis, prognosis, and recovery plan generation. Investigation of the information requirements of these tasks has shown that the determination of paths figures largely: what components or systems are connected to what others, how are they connected, whether connections satisfying certain criteria exist, and a number of related queries. The formulation of such queries frequently requires capabilities of the second-order predicate calculus. An information system is described that features second-order logic capabilities, and is oriented toward efficient formulation and execution of such queries.
Feyock, Stefan↗
Human Capabilities Assessments for Autonomous Missions: A Multi-Team Research Effort to Reduce Risk in the Human-System Integration Architecture for Future Deep-Space Missions
In future exploration missions beyond low earth-orbit, crew will have to execute complex operations and respond to off-nominal events, without real-time support from Mission Control. It is anticipated that increased reliance on automated systems, including human-centric vehicle and information architecture, will need to be designed to support the crew; increased risk to performance, health, and safety may occur if these are not implemented appropriately. The Human Factors and Behavioral Performance Element (HFBP) in the NASA Human Research Program supports research to characterize and mitigate such human health and performance risks, including the Risk of Adverse Outcome Due to Inadequate Human Systems Integration Architecture (HSIA). The HSIA risk addresses the integration of onboard capability and the crew roles and responsibilities necessary to enable the crew to respond effectively and efficiently in the increasingly autonomous mission operations environment. In 2017, HFBP released the “Human Capabilities Assessments for Autonomous Missions” (HCAAM) research topic to address HSIA related questions. HCAAM is a major NASA research effort that has assembled a multidisciplinary team from seven institutions to work closely with design and engineering efforts on research towards developing and refining human performance standards, guidelines and automation tools. The scientific focus is on quantitative assessment of human capabilities relevant to future deep-space missions during which earth/spacecraft communication is so delayed and intermittent that the crew must be able to function autonomously. The integrated strategy of the HCAAM project characterizes human capabilities and limitations related to potential performance decrements during long duration exploration mission spaceflight as relevant to both routine and complex task performance; defines system characteristics that reduce the likelihood or impact of potential decrements in human performance capabilities; performs integrated assessment of intelligent system responses within the context of an operational environment with relevant NASA tools, systems, and data structures in order to determine positive or negative interactions and validate recommended approaches; and proposes specific updates to existing standards and guidelines for inclusion in NASA handbooks for the design of future spacecraft intelligent systems that provide crew performance assessment/feedback, and to also serve as decision-support aids for the onboard crew (i.e., NASA-STD-3001, and NASA/SP-Human Integration Design Handbook (HIDH)). The scientific research vectors being addressed by the seven HCAAM teams include: - crew task performance (accuracy, efficiency) (crew + automation) - crew performance (accuracy, efficiency) - crew Situation Awareness - procedure design and multi-modal enhancement - concurrent tasking (mixed manual + some level of autonomy) - task handover - crew self-planning and time-lining - task design - trust in automation, real-time calibration - human multi-sensory feedback and guidance - human trust in on-board software-based intelligent assistants - virtual assistants The presentation will highlight plans and progress made in each of these research areas as well as the methods by which surrogate astronaut crews in the NASA JSC HERA spaceflight analog facility will function as human test subjects for all of the HCAAM research projects.
HCAAM VNSCOR↗
Sensory Motor Coordination in Robonaut
As a participant of the year 2000 NASA Summer Faculty Fellowship Program, I worked with the engineers of the Dexterous Robotics Laboratory at NASA Johnson Space Center on the Robonaut project. The Robonaut is an articulated torso with two dexterous arms, left and right five-fingered hands, and a head with cameras mounted on an articulated neck. This advanced space robot, now driven only teleoperatively using VR gloves, sensors and helmets, is to be upgraded to a thinking system that can find, interact with and assist humans autonomously, allowing the Crew to work with Robonaut as a (junior) member of their team. Thus, the work performed this summer was toward the goal of enabling Robonaut to operate autonomously as an intelligent assistant to astronauts. Our underlying hypothesis is that a robot can develop intelligence if it learns a set of basic behaviors (i.e., reflexes - actions tightly coupled to sensing) and through experience learns how to sequence these to solve problems or to accomplish higher-level tasks. We describe our approach to the automatic acquisition of basic behaviors as learning sensory-motor coordination (SMC). Although research in the ontogenesis of animals development from the time of conception) supports the approach of learning SMC as the foundation for intelligent, autonomous behavior, we do not know whether it will prove viable for the development of autonomy in robots. The first step in testing the hypothesis is to determine if SMC can be learned by the robot. To do this, we have taken advantage of Robonaut's teleoperated control system. When a person teleoperates Robonaut, the person's own SMC causes the robot to act purposefully. If the sensory signals that the robot detects during teleoperation are recorded over several repetitions of the same task, it should be possible through signal analysis to identify the sensory-motor couplings that accompany purposeful motion. In this report, reasons for suspecting SMC as the basis for intelligent behavior will be reviewed. A robot control system for autonomous behavior that uses learned SMC will be proposed. Techniques for the extraction of salient parameters from sensory and motor data will be discussed. Experiments with Robonaut will be discussed and preliminary data presented.
Peters, Richard Alan, II↗
Situation Awareness of Onboard System Autonomy
We have developed intelligent agent software for onboard system autonomy. Our approach is to provide control agents that automate crew and vehicle systems, and operations assistants that aid humans in working with these autonomous systems. We use the 3 Tier control architecture to develop the control agent software that automates system reconfiguration and routine fault management. We use the Distributed Collaboration and Interaction (DCI) System to develop the operations assistants that provide human services, including situation summarization, event notification, activity management, and support for manual commanding of autonomous system. In this paper we describe how the operations assistants aid situation awareness of the autonomous control agents. We also describe our evaluation of the DCI System to support control engineers during a ground test at Johnson Space Center (JSC) of the Post Processing System (PPS) for regenerative water recovery.
Schreckenghost, Debra↗
Capability Considerations for Enhancing Safety on Long Duration Crewed Missions: Insights from a Technical Interchange Meeting on Autonomous Crew Operations
As future flight crews on long duration deep space missions are expected to operate more autonomously, considerations must be given to onboard capabilities and human-computer teaming that will fortify the safety net traditionally provided by the Mission Control Center. In August 2018, the Human Factors and Behavioral Performance Element of NASA's Human Research Program convened a Technical Interchange Meeting (TIM) on Autonomous Crew Operations at NASA Ames Research Center to address how intelligent technologies can be utilized to augment crew capabilities to support real-time anomaly response. In this paper, we highlight three topic areas discussed at the TIM that have direct implications for future crew anomaly response capabilities: smart structures, cognitive assistants, and manpower.
autonomous crew operations↗
TruePAL – An AI Assistant for First Responder Safety
This paper presents the development of an AI assistant, Trusted and Explainable Artificial Intelligence for Saving Lives (TruePAL), to provide real-time warning of risks of potential crashes to the first responders. The TruePAL system employs an AI and deep learning technology for saving first responders and roadside crews lives in and around active traffic. A deep neural network (DNN) and a Non-Axiomatic Reasoning System (NARS) are implemented as an AI system. A mobile app with AI interface is developed to perform verbal communication with the first responders. The TruePAL team has developed an explainable AI approach by opening up the DNN blackbox to extract the activation filters of various features and parts of the targeted objects. The combination of DNN and NARS makes the TruePAL system explainable to the users. TruePAL ingests on-board cameras, radar, and other sensor signals, analyzes the environment and traffic patterns to generate timely warning to drivers and roadside crews to avoid crashes. The TruePAL team, in collaboration with the Miami/Dade Police Dept., has designed five use cases and multiple sub-scenarios in a CARLA driving simulator to test the capability of TruePAL in timely warning to the first responder drivers in potential crash scenarios. We have successfully demonstrated its capability of timely warning in over a dozen scenarios based on the use cases. The preliminary test simulation results show that TruePAL could provide the drivers and crew members advanced warning before a crash occurs.
Chow, Edward↗