Search NASA⌕ Search

SEARCH · Search NASA

Results for “Intelligent Crew Assistant”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

41 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Objective Structured Clinical Evaluation (OSCE) of an Artificial Intelligence (AI) Clinical Decision Support System (CDSS) Tool

BACKGROUND Objective Structured Clinical Evaluations (OSCEs) have long been established as a robust methodology for summative assessment of clinical skills and decision-making during medical education. The recent integration of Artificial Intelligence (AI) into clinical decision-making processes has prompted the need for novel evaluation frameworks to assess the efficacy and reliability of AI clinical decision support system (CDSS) tools. This abstract outlines the process of quantitatively evaluating a novel CDSS (“Doc in a Box” Google 2024) trained on curated medical spaceflight data in the psychomotor domain as it interfaces with a human volunteer acting as the crew medical officer (CMO). PURPOSE The AI CDSS under review was developed as part of the Lunar Command and Control Interoperability (LuCCI) project, which is intended to address a gap in how Lunar Surface Systems (LSS) would interoperate across multiple programs, commercial partners, and international partners. The project objective is to define, prototype, integrate, and evaluate an interoperable lunar command, control, data, and software reference architecture to enable autonomy and informatics capability through common standards across LSS. A multi-modal AI-based CDSS compatible with Federated LSS will assist clinicians in diagnosing and managing complex medical conditions by providing evidence-based recommendations through predictive analytics. Given the critical role of decision-support as NASA continues to evolve its Earth-independent medical operations (EIMO), it is imperative to ensure that such AI tools perform reliably and align with clinical standards during progressive lunar and Martian exploration class missions. METHODS The OSCE framework, traditionally used for evaluating human clinicians, was adapted to assess the AI tool's decision-making capabilities in simulated clinical scenarios. In this adapted OSCE, the AI CDSS was tested across a series of structured clinical scenarios designed to mimic real-life spaceflight patient cases. These scenarios included a range of conditions and complexities, allowing for comprehensive assessment of the tool's performance. Key evaluation metrics included accuracy of diagnosis, timeliness of decision-making, and appropriate recommendations for therapies. The OSCE was scored by human physician evaluators who assessed the AI's recommendations in comparison with expert clinicians' medical decision making to ensure alignment with best practices and the standard of care. RESULTS Preliminary results indicate that the AI CDSS demonstrated high accuracy in diagnostic recommendations and decision support across various scenarios. However, certain limitations were noted, such as occasional discrepancies in handling complex or nuanced cases that required a more contextual understanding. Additionally, the tool scored higher on the diagnostic portion of the rubric, with lower scores in the therapeutic recommendations. These findings highlight the importance of continuous refinement and validation of AI tools through rigorous evaluation frameworks like the OSCE. The adaptation of OSCEs for AI tools presents several advantages, including a structured and reproducible approach to evaluation, the ability to test AI systems in diverse clinical scenarios, and the opportunity to benchmark AI performance against established clinical standards to permit charting of future progress as aerospace medicine evolves as a discipline. Remaining challenges include ensuring that these evaluations capture the full spectrum of clinical decision-making scenarios that will be confronted by CMOs during missions and adequately reflecting real-world variability of the austere spaceflight environment. CONCLUSION Employing OSCEs to evaluate AI clinical decision support tools offers a promising approach to validating their clinical utility and efficacy. This methodology not only provides insights into the tool's performance but also fosters ongoing improvement and alignment with standard of care practices. Future research should focus on refining these evaluation processes and addressing limitations to enhance the integration of AI tools in clinical spaceflight settings. REFERENCES Scott S, Hearns V, Barker MA. Testing Clinical Skills: A Look at the OSCE and USMLE Clinical Skills Exams. S D Med. 2019 Oct;72(10):451-453. Majumder MAA, Kumar A, Krishnamurthy K, Ojeh N, Adams OP, Sa B. An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives. Adv Med Educ Pract. 2019 Jun 5;10:387-397. Karam VY, Park YS, Tekian A, Youssef N. Evaluating the validity evidence of an OSCE: results from a new medical school. BMC Med Educ. 2018 Dec 20;18(1):313.

Ariana M Nelson↗
NASA NTRS2022

Supporting Hazard Analysis for Wildfire Response Using fmdtools and MIKA

The System Wide Safety (SWS) Safety Demonstrator (SD) Series drives development of an increasingly capable In-Time Aviation Safety Management System (IASMS) focusing on humanitarian applications, starting with wildfire response (SD-1). The goals of this report are to (1) provide an early hazard analysis and mitigation evaluation of wildfire response to support these efforts and (2) provide a demonstration of capabilities of the Fault Model Design Tools (fmdtools) and Manager for Intelligent Knowledge Access (MIKA) tools. fmdtools provides a modeling, simulation, and resiliency analysis framework in which a wildfire response model, the System Modeling and Analysis of Resiliency in Scalable Traffic Management for Emergency Response Operations (SMARt-STEReO), is built. MIKA is an intelligent knowledge manager with several capabilities, including assisting in hazard analysis by extracting and analyzing hazards from historical incident reports. The following topics are covered in the report: Understanding Wildfire Hazard Dynamics. We provide a description and simulated examples of how hazards occur in the SMARt-STEReO model of wildfire response and their effect on its outcome. This provides a common mental model and focuses the analysis presented in the remainder of the report. Wildfire Hazard Identification. MIKA identifies wildfire hazards from three relevant datasets: the ICS-209-PLUS, SAFECOM, and SAFENET. Hazards are manually organized into a taxonomy and MIKA analyzes each hazard’s effects, likelihood, severity, and risk. Evaluating Mitigation Strategies. The SMARt-STEReO wildfire response model built in fmdtools evaluates a subset of identified hazards. Specifically, we simulate the effect of communications faults and equipment faults on operator safety, the effect of changing winds and flammability, and a scenario with multiple ignition points and heavy smoke. Tool Limitations and Usage Considerations. We provide a discussion of appropriate tool use cases as well as limitations and considerations for usage. The tool findings are used to synthesize recommendations for wildfire response operations, which can be captured as part of an IASMS. Key recommendations are as follows: Hazards are identified from a broad spectrum of sources including aircraft subsystems, operational sources, and ground crew operations. Highest risk operational environment hazards identified are Evacuations. The highest risk manned aerial operations hazard categorized is Jumper Operations Mishap. Ground crew hazards that are highest risk are Burns, Cargo Operations Overhead, Dehydration, Entrapment, Falling Objects, Heart Attacks, Heat Exhaustion, Inadequate Training or Certification, Vehicle Breakdown, and Vehicle Collision. Modelled containment failures arise from a mismatch between the difficulty of the firefighting scenario and the capacity (e.g., speed, effectiveness, awareness) of the response. In firefighting scenarios where containment is possible (e.g., because the fire does not spread too quickly), these mismatches can occur because of a change in environmental conditions (e.g., wind, flammability, etc) or because of planning, equipment, or communications faults. Improvements to communications increase the capacity of the firefighting response by reducing the time needed to respond to the fire. While surveillance does not increase this capacity by itself, it increases operator safety by increasing state awareness, enabling firefighters to evade approaching fires. Increasing both has a synergistic effect. In general, these performance and resilience increases generalize over fault scenarios as well as unforeseen changes to circumstances (i.e., wind, aridity, etc.). However, these improvements need to be designed so as not to make the system prone to persistent large-scale communications outages, which can reduce performance.

Hazard analysis↗
NASA NTRS2004

Human-Robot Planetary Exploration Teams

The EVA Robotic Assistant (ERA) project at NASA Johnson Space Center studies human-robot interaction and robotic assistance for future human planetary exploration. Over the past four years, the ERA project has been performing field tests with one or more four-wheeled robotic platforms and one or more space-suited humans. These tests have provided experience in how robots can assist humans, how robots and humans can communicate in remote environments, and what combination of humans and robots works best for different scenarios. The most efficient way to understand what tasks human explorers will actually perform, and how robots can best assist them, is to have human explorers and scientists go and explore in an outdoor, planetary-relevant environment, with robots to demonstrate what they are capable of, and roboticists to observe the results. It can be difficult to have a human expert itemize all the needed tasks required for exploration while sitting in a lab: humans do not always remember all the details, and experts in one arena may not even recognize that the lower level tasks they take for granted may be essential for a roboticist to know about. Field tests thus create conditions that more accurately reveal missing components and invalid assumptions, as well as allow tests and comparisons of new approaches and demonstrations of working systems. We have performed field tests in our local rock yard, in several locations in the Arizona desert, and in the Utah desert. We have tested multiple exploration scenarios, such as geological traverses, cable or solar panel deployments, and science instrument deployments. The configuration of our robot can be changed, based on what equipment is needed for a given scenario, and the sensor mast can even be placed on one of two robot bases, each with different motion capabilities. The software architecture of our robot is also designed to be as modular as possible, to allow for hardware and configuration changes. Two focus areas of our research are safety and crew time efficiency. For safety, our work involves enabling humans to reliably communicate with a robot while moving in the same workspace, and enabling robots to monitor and advise humans of potential problems. Voice, gesture, remote computer control, and enhanced robot intelligence are methods we are studying. For crew time efficiency, we are investigating the effects of assigning different roles to humans and robots in collaborative exploration scenarios.

Tyree, Kimberly↗
NASA NTRSDate not supplied

CLINICAL DECISION SUPPORT: PATH TO FUNCTIONAL REQUIREMENTS

Long-duration, deep-space exploration missions present significant challenges to crew health and performance. These challenges include the individual and combined effects of microgravity, radiation exposure, isolation, limited resources (mass, volume, power, data and crew time), limited options for evacuation and those associated with delayed or constrained communications, all of which demand greater crew autonomy. Specifically, as the communication delays intensify the further we explore space, the unqualified need for Earth-independent medical operations focused on autonomous diagnosis, treatment and prevention will be key to mission continuation and success. To augment the requisite knowledge, skills and abilities (KSAs) of a time-constrained crew operating under stressful conditions, combatting fatigue, and facing a potential medical crisis, a robust clinical decision support system (CDSS) is a probable solution that would facilitate, guide and inform Earth-independent medical operations, while assisting crewmembers through various clinical presentations. The Exploration Medical Capability (ExMC) Element of the Human Research Program (HRP) is expanding the boundaries of space medical systems to advance the care of astronauts on future exploration missions beyond low Earth orbit. ExMC is actively identifying and testing next-generation medical care and crew health maintenance technologies. The Clinical Decision Support (CDS) project addresses gap Medical-701 within the Inflight Medical Conditions risk: “Enhance medical capabilities within an exploration medical system.” Though mass, volume, and power will face increasing constraints, the projected computational capabilities of spacecraft systems will increase exponentially as information technology continues to advance this decade and beyond. Hence, data, software and computational resources will play an essential and synergistic role in maintaining crew health, wellness and performance in deep space missions. The focus of the CDS project is to develop recommended requirements for an in-vehicle CDSS that acts as a ‘virtual assistant’ for delivering optimal health, performance and medical care during exploration missions. The CDSS is envisioned as an integrated, software-based tool deployed on a laptop computer or handheld device. The CDSS will assist the crew and ground support when interacting with knowledge/databases (e.g. records, pharmacy, schedule), instrumentation (e.g. imaging, physiological monitoring devices), and habitat (e.g. wellness system, task performance system) and vehicle systems (e.g. environmental system, communication system). In addition, the human interface will employ a context-based approach that accounts for the crew’s situation. Thus, extraneous and clinically/operationally non-relevant information are reduced to avoid an increase in cognitive load. The framework of an ideal spaceflight CDSS is to include core and advanced analytical features that incorporate work from collaborators yet maintain a flexible platform for integrating new technology in the future. In fiscal year 2021 (FY21), the CDS project identified requirements through two primary mechanisms: (i) the development of software implementation prototypes and (ii) the application of systems engineering processes. The CDS project developed and tested a series of increasingly complex system prototypes that were based on use cases derived from the CDSS concept of operations (ConOps). These software implementations yielded insights on CDSS functionality as well as lessons learned that provided the initial requirements for CDSS capability. By applying a systems engineering (SE) approach, medical scenarios provided in the ConOps and the use cases for software implementation underwent functional decomposition to identify CDSS functionality. Also, systems-based modeling language (SysML) tools such as activity diagrams were developed from the same ConOps and use cases to identify CDSS functionality. The lessons learned from software implementation defined both specific requirements and broad areas of requirements. Within these defined broad requirement areas, further analysis of the SE products identified specific capability that resulted in the final functional requirements. In summary, the software prototypes, functional decomposition of the ConOps and use cases, and SysML diagrams provided the basis for the CDSS requirements developed in FY21. In the upcoming year, these requirements will be refined for their final ExMC baseline review in latter FY22.

clinical decision support↗
NASA NTRS2018

NASA Pilot-Engaged Expert Response Using IBM Watson Technology: Prototype Evaluation of Knowledge Retrieval System

NASA Langley Research Center and IBM have been investigating the use of IBM Watson technology in aerospace research and development. One application of Watson technology is the Pilot-Engaged Expert Response (PEER) use case. The PEER system is envisioned as an in-cockpit advisor that will act as a source of situationally-relevant information for pilots and other flight crew members to assist in decision making about real-time events and situations that arise in the course of aircraft operations. PEER will make available vast stores of knowledge and information quickly and directly, putting important informational resources where they are needed most. IBM has worked with NASA to develop an architecture and articulate a roadmap for the development of the PEER system. That vision is built around Watson Discovery Advisor (WDA) software solution, derived from IBM's Jeopardy!-winning automatic question answering system. PEER makes use of WDA's sophisticated question-answering capabilities as its core, adding important User Interface components and other customizations for the cockpit environment, including communication with flight systems and other external data sources. The development plan for PEER includes four development stages, with the current project constituting the first phase. In this project, a prototype instance of PEER was successfully adapted to the aviation domain, enabling users to ask questions about aviation topics and receive useful and accurate answers to these questions. Major tasks accomplished include the development of procedures for domain adaptation through automatic lexicon extraction from domain glossaries; generation of question-answer training data which was used to train the system; and assessment of the effectiveness of domain adaptation, which showed a dramatic improvement in the ability of the PEER system to answer domain-relevant questions. In addition, the vision for the PEER system was pushed forward by the articulation of a plan for the automatic enhancement of question-answering with contextual information. This initial phase focused on two main goals: 1) the targeted domain adaptation of the underlying WDA system to the aviation domain; and, 2) the design of the software systems needed to leverage flight-contextual data. Domain adaptation of the WDA system proceeds via three main activities: Domain data ingestion, lexical customization and model training. A textual corpus consisting of 1,147 individual documents with more than 7.5 million words of text was ingested into the system and this served as the basis of all further development. A domain lexicon of over 3,500 aviation-domain terms was semi-automatically generated from domain documents and used to train the system. In addition, a set of over 500 question-answer (QA) pairs relevant to the PEER use case was developed; these were used to train and assess the system. These important first steps established the basis for the PEER system. In addition, steps were taken towards the integration of the PEER system into the cockpit environment with the development of a functional design for the Contextual Data Augmentation (CDA) subsystem. This subsystem brings to bear contextual data to improve system responses. It has three main submodules: the Contextual Data Collection module, the Contextual Data Selection module, and the Contextual QA Augmentation module. These modules form a processing pipeline that addresses the problems associated with automatically integrating information from external resources into the knowledge-retrieval mechanism.

Machine learning↗