SEARCH · Search NASA

Results for “large language model (LLM)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Problem Complexity and LLM: H-M Team Reliability in Challenging Environments

In traditional human-machine operations, the functional decomposition of actions and responsibilities among various agents is assigned a priori. For instance, in current air traffic operations, although assisted by software, human pilots have the ultimate control of aircraft. Multiagent human-machine and machine-machine systems will face problems of varying and potentially unpredicted complexity in future challenging scenarios of planetary, en route, and orbital activities. Hence, it is important to enable dynamic transfer of decision-making to appropriate team members, human or machine, depending on which agent is best equipped to solve that specific problem on a time budget. In this paper, we consider aspects of problem-solving and its modeling that affect the outcomes of decision-making as a function of solution quality and the likelihood of solving the problem on a required time budget. We focus on Large Language Models (LLM) as potential machine teammates and conclude that practical, predictive modeling of their performance, at the current stage of their development, is infeasible. Simple examples help us illustrate that current LLM will require fundamental advancements to provide reliable support in team decision-making, especially in safety-critical and time-critical domains. The study is not meant to diminish the value of the remarkable capabilities of LLM, but rather to gain a better understanding of the technology’s appropriate use and the needed additions.

function allocation↗

NASA NTRSDate not supplied

Trust-Informed Large Language Models via Word Embedding-Knowledge Graph Alignment

A major weakness of a Large Language Model (LLM) is its tendency to accept information at face value, often leading to injection of erroneous information and inducing a greater probability of hallucinating non-existent information. While Retrieval Augmented Generation (RAG) uses external knowledge sources to bolster LLMs through grounded truth, this work seeks to explore methods to engender a LLM with an intrinsic capability to evaluate an input’s believability without relying on external knowledge sources. We investigate unifying a LLM with a Knowledge Graph (KG) and using the KG to reinforce the LLM’s internal word embedding while also maintaining belief metrics along the edge’s in the KG.

Large Language Model↗

NASA NTRS2024

Artificial Intelligence Medical Support for Long-Duration Space Missions

We envision an artificial intelligence (AI) based system that will provide support and recommendations to the crew medical officer (CMO) and ground flight surgeon during long-duration space missions. Such a system would be pretrained on the knowledgebase of clinical knowledge on Earth, minimizing the amount of Earth data that needs to be transferred into space. Then during deployment, the system would be constantly refined through active learning from diverse streams of data from sensors in the spacecraft, data collected daily from individual astronauts, and human-in-the-loop feedback from the crew. The model could be interrogated for predictions and recommendations on personalized crew health based on the overall status of the spacecraft, medicinal stores, and status of other crew members. Adaptation techniques would be used to incorporate spaceflight data that have very different distributions from the training data due to the extreme environment. Edge computing and the most advanced neuromorphic processing would enable computation in scenarios with low power and bandwidth, while dimensionality reduction would be employed to ensure that the input data streams from spaceflight are as small as possible. In order to realize this long-term vision, several hardware and software aspects need to be developed and assembled. First, models pretrained on Earth biomedical data would need to be evaluated for predictive accuracy, and the best one selected. That model would need to be adapted to learn from diverse, sparse, and inconsistently measured data streams, as well as human-in-the-loop feedback. A data integration, standardization, and dimensionality reduction methodology would need to be developed to handle all data types and feed them into the model. Once the software and data infrastructure is developed, it would need to be integrated with small footprint compute processors and tested in high-radiation, high-vibration, unregulated temperature situations. As a short-term goal, we recommend to focus on the development of the data and model software structure. Several large language models (LLM) already exist that have been trained on Earth biomedical and clinical knowledgebases, including BioMedLLM, Med-PaLM, SPOKE LLM, and Foresight. These models need to be evaluated for accuracy and the best one chosen for a proof-of-concept structure, while maintaining awareness of the accelerating AI field and incorporating any newly improved model architectures as needed. Then, we recommend to develop a database of synthetic data types to mimic the diverse data streams that are expected in a long-duration space mission. This should include environmental and microbial data from the spacecraft, non-invasive data from wearables and point-of-care devices employed by astronauts, and more invasive molecular and physiological monitoring of clinical and biomarker data from astronauts. The data standardization methodology should be developed, and these data streams used to refine the clinical LLM. Several scenarios should be developed that could plausibly come up in a long-duration space mission, and changes or aberrations introduced to the data at specific times to mimic these scenarios. Then, question and answer tasks should be designed to interrogate the model for predictions and recommendations, with acceptable answers already identified.

Artificial Intelligence↗

NASA NTRSDate not supplied

Harnessing Artificial Intelligence for Medical Diagnosis and Treatment During Space Exploration Missions

From May 8th to June 9th, 2023, I had the opportunity to participate in an experiential learning experience at Johnson Space Center in Houston, TX with Exploration Medical Capability (ExMC), an element of the NASA Human Research Program. During this research experience, I was not only able to work on the above titled research project, but also gain an immense exposure to the field of aerospace medicine, make numerous connections within the field, tour NASA facilities, as well as travel to the Aerospace Medical Association Annual Conference (AsMA) in New Orleans. To briefly introduce my project, it is well understood that the medical capabilities available to crew medical officers (CMOs) on the International Space Station will be different than the capabilities available and needed during deep space exploration missions to the Moon, Mars, and beyond. Ground support is particularly limited due to distance, communication delays (or lack of communication), and lack of resupply. Therefore, to support medical care by CMOs on these missions, robust clinical decision support systems (CDSSs) must be designed. The recent publication and public launch of generative artificial intelligence (AI) tools based upon large language models (LLM) such as ChatGPT provides the opportunity to create a smart assistant for onboard triage, diagnosis, and treatment of medical conditions. Ultimately, the overall purpose of the project was to research what AI tools currently exist or are in development, and to see how they might be implemented onboard during exploration class spaceflights of the future. The ExMC element is actively developing several tools to be used in preparation for and during deep space exploration missions. One of those tools, known as IMPACT, is a probabilistic risk assessment model which can be used to propose a desired medical system (based on mass and volume) and suggest the clinical outcomes likely to occur for a design reference mission (DRM). The group recently presented the IMPACT model and a DRM of interest titled “Modified Long Duration Lunar Orbital and Lunar Surface” (mLDLOLS) at the recent AsMA conference. The mLDLOLS mock mission is a 9 month and 6-day deep space exploration mission consisting of time in Moon’s orbit (3 months on the Gateway space station), on the lunar surface (3 months within habitat), and another 3 months on Gateway before return to Earth. For this DRM, IMPACT ultimately outlined a preferred medical system that was then associated with medical conditions considered to be most likely based on frequency, most likely to cause astronaut task time loss (TTL), most likely to cause return to definitive care (RTDC), and most likely cause loss of crew life (LOCL). IMPACT also highlighted the medical capabilities/skills that would be required to care for those medical conditions, such as performing a history of present illness or musculoskeletal exam with ultrasound. The primary objective of the project was to perform a survey of the AI tools and systems applicable to the conditions outlined for the proposed mLDLOLS mission. Using PubMed (including most relevant MeSH terms) and Google Scholar, we then created a robust annotated bibliography organized by condition. The 56-page and over 500 reference annotated bibliography was subsequently used to create a review outline that would become the basis for drafting of a future publication. For the review outline, we took those medical conditions researched within the annotated bibliography (condition-based approach) and deployed a systems-based approach, combining those medical conditions and related tools into ten categories. These categories included general/all-purpose CDSSs, tools to diagnose or manage respiratory, dermatologic, neurologic, auditory and vestibular, ophthalmic, musculoskeletal, infection-associated, and gynecologic conditions, as well as tools that could be deployed in the setting of trauma/emergency. With the completion of the 30-page outline, we then began drafting the review paper. To conclude the research experience, I presented the findings from our survey to the ExMC Clinical and Science team. With these objectives, I ultimately learned about the number of AI tools that exist today to assist medical professionals with the triage, diagnosis, and management of several medical conditions. These tools can span from chatbot assistants to help triage knee pain to vision transformer models that can identify ophthalmic conditions based on ocular surface images captured with a cell phone. We also highlighted the current gaps that exist in the literature alongside the advancements that are needed to make the desired CDSS for deep space exploration missions. With this experience, I certainly confirmed an existing career goal and identified several additional skills needed to become an aerospace medical doctor including knowledge of critical care in an extreme medicine setting, aerospace engineering and human integration systems, artificial intelligence, machine learning, and risk models. I also identified numerous transferable skills for this career goal including the basic knowledge of medicine (MD), deployment of the scientific method for critical thought about new scientific questions (PhD), review of published literature, including creating an annotated bibliography (PhD), as well as detailed scientific writing (PhD). The results of my research will likely guide the design of an all-encompassing onboard medical assistant for use during deep space exploration missions of the future. I plan on sharing the outcomes from this experience with my peers at a student seminar in the Fall semester on August 30th. During the seminar, I will detail the project, my experience at NASA and AsMA, as well as offer best practice guidelines for students entertaining similar experiences or careers. In conclusion, I would like to thank the WVU School of Medicine, Research and Graduate Education office, as well as NASA ExMC for the unwavering support of this life-changing experience.

Ryan A. Lacinski↗

NASA NTRSDate not supplied

Harnessing Artificial Intelligence for Medical Diagnosis and Treatment During Space Exploration Missions

BACKGROUND The medical capabilities necessary for long-duration exploration missions (LDEMs) will differ tremendously from those currently available to crew medical officers (CMOs) on the International Space Station (ISS). Ground support will be more challenging due to distance-related communication delays and data transmission, and resource utilization must be optimized given limited ability for resupply. Clinical decision support systems (CDSSs) can help mitigate these limitations. The recent launch of generative artificial intelligence (AI) tools based upon large language models (LLM) support the creation of a smart assistant for onboard triage, diagnosis, and guided treatment of medical conditions during these missions. The Informing Mission Planning via Analysis of Complex Tradespaces (IMPACT) tool can help predict which clinical problems and outcomes are likely to occur for a design reference mission (DRM) and assist Medical Operations and systems engineering teams in creating a medical system that may optimally mitigate the predicted risks. The purpose of this study was to identify AI tools currently available or in development for the assistive diagnosis and care of medical conditions predicted for an extended duration Lunar mission. METHODS The 119 medical conditions currently built into the IMPACT suite were categorized into systems, and these diagnoses were used as keywords for our literature search. Using PubMed and Google Scholar, we performed a literature survey of AI tools applicable to these conditions. Article inclusion criteria included publication between the years 2017-2023, as the sentinel paper discussing the “selective attention” driving ChatGPT and other generative transformer models was published in June 2017. Where applicable, we reviewed only the top 1000 research articles (based on relevance) for each of the keywords/phrases. AI tools whose training sets were exclusive to a pediatric patient population were excluded. We also excluded any medical diagnostic tools (such as CT, MRI, mass spectrometry) or procedures (such as endoscopy, surgery) that are unlikely to be available during LDEMs due to mass and volume constraints, CMO knowledge, skills, and abilities, and/or inherent procedural risks. RESULTS Our survey highlighted several AI-driven tools for the triage, diagnosis, and management of those medical conditions highlighted by IMPACT. Selected publications for each medical condition were then screened for inclusion within ten systems-based categories including: general diagnostic tools (25), tools to diagnose and manage respiratory (40), dermatologic (34), neurologic (28), auditory and vestibular (30), ophthalmic (34), musculoskeletal (104), infection-associated (92), and gynecologic (19) conditions, as well as tools that could be deployed in the setting of trauma and emergency (34). CONCLUSIONS Numerous AI-driven tools were highlighted within this literature survey, ranging from chatbot assistants that triage knee pain to vision transformer models for diagnosis of ophthalmic conditions using ocular surface images captured with a mobile phone. Remaining challenges include optimizing connectivity and integration of existing and developing systems into the vehicles or habitats. Notably, findings from this survey could help guide the initial design of an all-encompassing, onboard medical AI assistant for use during future LDEMs.

R A Lacinski↗

NASA NTRS2024

Analyzing EOSDIS Dataset Research Outputs using Knowledge Graphs and Large Language Models

Datasets, unlike publications, can be updated over time, with each new version receiving a DOI but not always being linked to previous ones. This complicates tracking citations across a dataset’s lifecycle. We address this by integrating dataset versions and citations into a knowledge graph (KG), which helps trace dataset citations and analyze dataset usage in applied research. To categorize publications from various journals, we fine-tuned NASA IMPACT INDUS Large Language Model (LLM) on a labeled publication set, assigning publications to one of twenty applied research areas. By linking datasets to these research areas, we improved dataset searchability and discovery through these domains.

open-source↗

NASA NTRSDate not supplied

Discovering Research Areas in Dataset Applications Through Knowledge Graphs and Large Language Models

Scientific datasets are increasingly cited in peer-reviewed journal publications, facilitating easy access to research utilizing those datasets. Datasets undergo a life cycle where older versions of datasets are replaced by newer versions often due to improvements in data resolution, algorithms, and other factors. Unlike peer reviewed documents registered with a single Digital Unique Identifier (DOI), datasets can be updated over time and the newer version of the datasets are registered with a new DOI which is not necessarily linked to the previous version of the dataset. It is challenging when publications citing a dataset need to be traced over the entire life cycle of that dataset. We provide an innovative approach to link the dataset versions and publications using a knowledge graph (KG). KG can help to trace the dataset cited in publications over the entire dataset life cycle and shed light into dataset usage in various applied research areas. We fine-tuned the pretrained NASA IMPACTINDUS Large Language Model (LLM) on a set of labeled publications abstracts. Our results showed that 87% of the publications were classified into one of twenty applied research areas, while the remaining 13% were classified into non-applied research areas. By linking datasets to applied research areas through the KG and employing Global Change Master Directory(GCMD), a well-established controlled vocabulary of scientific keywords describing Earth science datasets, we contribute to a transparent and advanced search and discovery mechanism for datasets across the Earth data ecosystem. The integrated KG and LLM approach is now incorporated and operational in dataset publication management at one of NASA’s Earth science data archival centers.

data provenance↗

NASA NTRSDate not supplied

Enhancing Dataset Discovery and Usage Tracking in Earth Sciences: Integrating Knowledge Graphs and Large Language Models

NASA's Data Active Archive Centers (DAACs) have played a crucial role in supporting a wide range of applied research in Earth and Environmental sciences. To date, over 20,000 publications have been collected, citing more than 3,000 NASA Earth science datasets. We present an innovative approach that links datasets and collected publications through a knowledge graph (KG). This KG enables the tracking of dataset citations throughout the dataset's lifecycle, revealing patterns of dataset usage across various applied research areas. We fine-tuned the pre-trained NASA IMPACT INDUS-Base Retriever Large Language Model (LLM) using a set of labeled publication abstracts. Our results indicate that 87% of the publications were classified into one of twenty applied research areas, while the remaining 13% were categorized into non-applied research areas. The classified publications linked to datasets are used to discover datasets by users interested in specific applied research and by dataset providers to determine dataset usage for applications.

open-source↗

NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or aerial wildfire operations reports to better understand the risks present. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement. These applications can benefit from the use of state-of-the-art natural language processing techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of adaptation of NLP tools to the aviation domain by fine-tuning transformer based models using aviation data sets. In 2018, a novel language model based on neural units (also called transformers) was created and became known as “Bidirectional Encoder Representations from Transformers” or BERT. This architecture combined with large amounts of English training data and innovative semi-supervised training tasks set the standard for what would later emerge as Large Language Models. The performance of these models was further improved by hyperparameter tuning and refinement of the semi-supervised training task and resulted in “Robustly Optimized BERT Pre-training Approach through hyperparameter tuning” or RoBERTa models. These pre-trained Large Language Models proved to be useful for a wide variety of natural language processing tasks such as text classification and question answering through a process called fine-tuning. The transformer architecture with pre-trained weights served as the basis with the last few layers replaced with layers fine-tuned to perform a new task e.g., a layer that provides a label for the entire input text. This process of fine-tuning can also be used to adapt the models to new domains; e.g., BioBERT started with the pre-trained BERT model and was completed by additional fine-tuning and training on biomedical documents. Transformer-based architectures can also be used to create rich representations of text called embeddings which can serve as the input to other machine learning models. This allows simpler algorithms such as logistic regression to use context-rich representations of the text while still remaining quick to train and evaluate. In the world of aviation, there is a growing demand for natural language processing and understanding but the domain presents unique challenges. Due to the technical content (and specialized language) of most aviation documents, fine-tuning pre-trained Large Language Models to specific tasks has not met the benchmark on natural language processing tasks set by simpler models trained from scratch on the data. To address this deficiency, this paper evaluates the improvements from fine-tuning a Large Language Model on a large set of aviation documents using the original semi-supervised training tasks before performing specific natural language tasks. In fine-tuning, a domain-specific dataset is used on the original training task but with the pre-trained Large Language Model instead of starting from a random initialization. This approach allows the model to be adapted to the specific domain language without discarding the information gained from training on general English data. This paper utilized two major dataset types to train and assess the RoBERTa fine-tuning performance. The first are 7,057 Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the semi-supervised fine-tuning. The second type is the 494 document classification labels to be used for evaluation. This down-stream evaluation aims to show the performance of the fine-tuned model, better understand how much data is needed for an effective fine-tuning, and how fine-tuning can be adapted for different applications in-the domain. After semi-supervised training, evaluation begins by encoding the documents for classification using the fine-tuned RoBERTa model. Then a logistic regression classifier is trained to label the document type and compared against our ground truth labels. This currently leads to a 82.8% accuracy on 10-fold cross validation showing improvement over baseline RoBERTa which achieved 81.0%. We plan to measure the improvements on additional tasks and it is expected that these improvements will lead to more robust models that can tackle the natural language processing challenges present in aviation datasets.

ATM↗

NASA NTRSDate not supplied

LSKnowledge: Nexus for Transformative Scientific Discoveries and Enhanced Information Retrieval in NASA Life Sciences Portal

We stand at the brink of an extraordinary transformation in the field of AI, driven by the convergence of generative AI and semantic technologies (e.g., knowledge graphs). This fusion holds immense potential and could redefine the future of scientific exploration, particularly in the realm of life sciences research. In this context, we shed light on the pivotal roles that Large Language Models (LLMs) and semantic technologies will play in advancing research, unearthing and comprehending life sciences information through innovative approaches, and empowering researchers to extract insights from NASA's extensive Life Sciences Data Archive. Within the NASA Life Sciences Portal (NLSP), the integration of LLMs and semantic technologies unlocks several advanced capabilities. First and foremost, it equips scientists with sophisticated tools to manage the ever-expanding wealth of scientific literature and data. Furthermore, it facilitates the creation of knowledge graphs that visually represent intricate relationships among biological entities, enabling comprehensive systems-level analysis. Additionally, the fusion of generative AI (including LLMs) and semantic technology can significantly benefit NASA's life sciences research by enhancing information retrieval and hypothesis generation. These tools enhance natural language understanding, facilitating knowledge discovery within NLSP. The overarching vision is to establish a cohesive knowledge ecosystem within NLSP, harnessing the power of LLMs and semantic technologies to synthesize and cross-reference data from diverse missions, disciplines, and research domains. This holistic approach ultimately deepens our understanding of how space environments impact life sciences data. To advance this initiative, we have launched LSKnowledge, aimed at enhancing the information retrieval capabilities of NLSP. In the short term, our primary goal is to develop a robust semantic search system. This system will empower HRP (Human Research Program) researchers to navigate NLSP data repositories more efficiently and precisely, catalyzing the process of hypothesis formation and scientific breakthroughs. To achieve this, we have employed pre-trained LLMs as part of a semantic search tool that can rank and highlight the most relevant records for user queries. To assess the tool's performance, we have curated a set of approximately 200 queries from subject matter experts (SMEs) and manually ranked the top records retrieved by both the current search system and the new semantic search, using SME judgments as the gold standard for relevancy. Herein, we present the results of our comparative analysis and illustrate how these findings have informed the fine-tuning of the system for enhanced performance. In the long term, our objectives include 1) retrieving publicly available information and integrating it with NLSP data to provide more precise answers to user queries, and 2) incorporating non-textual information from the NLSP database into our approach. In conclusion, the fusion of LLMs and semantic technologies within NLSP represents a pioneering stride towards reshaping the landscape of scientific discovery. This synergy not only equips researchers with powerful tools to navigate the burgeoning sea of information but also facilitates a deeper understanding of complex biological relationships, all while accelerating hypothesis generation and knowledge discovery. Through our initiative, LSKnowledge, we are committed to continually refining and expanding these capabilities, with the aim of not only enhancing information retrieval but also integrating diverse data sources to provide more precise insights. In the grand vision, NLSP strives to become the cornerstone of a comprehensive knowledge ecosystem, unraveling the enigmatic intricacies of life sciences phenomena in the context of space environments.

Life Sciences↗

NASA NTRSDate not supplied

Harnessing Large Language Models for Scientific Endeavors

The rapid proliferation of Large Language Models (LLMs) such as GPT, Bard, and Llama has revolutionized various sectors, including the scientific community. These models, with their potential to automate and augment tasks, are increasingly being recognized as both a valuable asset and a potential challenge in the realm of scientific research and data management. However, the current LLMs, primarily trained on general corpora, exhibit a limited understanding of scientific concepts and terminologies due to the lack of scientific corpus in their training data. Recognizing this gap, several groups are now advocating for the development of LLMs specifically tailored for scientific applications. A notable initiative in this direction is the Large Language Model effort initiated by NASA's CSDO. This endeavor aims to align LLM efforts across NASA’s Science Mission Directorate, develop a science-specific corpus and validation test set for model training, and create an encoder-only model for various downstream tasks. Moreover, the initiative also plans to develop a decoder-only model to explore the potential benefits and risks associated with a generative LLM for science. Lastly, the project aims to create a science evaluation suite, encompassing various categories of downstream scientific tasks, to serve as a benchmark for assessing the value of any LLM for future use. This presentation will provide an overview and current status of this ongoing initiative, highlighting its potential to reshape the use of LLMs in the scientific domain.

Rahul Ramachandran↗

NASA NTRSDate not supplied

Retrieval-Augmented Generation and LLM Agents for Biomimicry Design Solutions

We present BIDARA, a Bio-Inspired Design And Research Assistant, to address the complexity of biomimicry – the practice of designing modern-day engineering solutions inspired by biological phenomena. Large Language Models (LLMs) have been shown to act as sufficient general purpose task solvers, but they often hallucinate and fail in regimes that require domain-specific and up-to-date knowledge. We integrate Retrieval-Augmented Generation (RAG) and Reasoning-and-Action agents to aid LLMs in avoiding hallucination and utilizing updated knowledge during generation of biomimetic design solutions. We find that incorporating RAG increases the feasibility of the design solutions in both prompting and agent settings, and we use these findings to guide our ongoing work. To the extent of our knowledge, this is the first work that integrates and evaluates Retrieval-Augmented Generation within LLM-generated biomimetic design solutions.

biomimicry↗

NASA NTRSDate not supplied

Retrieval-Augmented Generation and LLM Agents for Biomimicry Design Solutions

biomimicry↗

NASA NTRSDate not supplied

Developing Concepts of Operations Using Multi-Step Tool Techniques With Large Language Models

The National Aeronautics and Space Administration (NASA) Air Mobility Pathfinders (AMP) project is developing and evaluating concepts of operations (ConOps) for safe, secure, and scalable Urban Air Mobility (UAM) operations. The AMP project’s Operational Concepts, Architecture, and Requirements Integration (OCARI) Team is using a Model Based System Engineering (MBSE) approach for integration, interoperability, and traceability of Advanced Air Mobility (AAM) ecosystems centered around urban air taxi services. The team’s goal is to define structures and behaviors needed for system feasibility, readiness, and interoperability, establish a UAM knowledge base, and trace and validate assumptions and requirements relevant to AAM. NASA Langley Research Center (LaRC) is spearheading an innovative digital engineering approach to integrate, communicate, and facilitate the research of multi-modal transportation systems. The Knowledge-based Digital Platform (KbDP) is a concept being developed that ties the workflows of Project Managers (PM), Principal Investigators (PI), and System Engineers together across organizational boundaries. It does so through the management of an information database defined by mathematical, data science, and system engineering principles. Machine Learning (ML) algorithms play a key role in this concept by extracting meaningful knowledge from relational and graph databases, document repositories, and system artifacts, which the human user leverages to greatly improve the efficiency and effectiveness of their research. Recent advancements in the field of Large Language Models (LLMs), specifically models trained for tool use, such as Command-R , now allow for the reliable implementation of single-step and multi-step tool-centric systems. These techniques provide the LLM with a set of tools, in our case Python functions, that can be called on to answer a much wider range of questions compared to LLMs implemented using a traditional single-source or Retrieval Augmented Generation (RAG) approach. Through this method, the LLM can pull information from multiple data sources, such as relational or graph databases, document repositories, application programming interfaces (APIs), and SysML artifacts depending on the user’s question. The LLM can also output the information in a variety of different formats, using output generation tools, such as CSV, UML, or SysML artifacts. Additionally, tools can be assigned roles and can work together to provide answers to queries in an “agent” like approach, similar to that implemented by Microsoft’s AutoGen framework where different agents can converse with each other to accomplish tasks. Previously, our team developed a chatbot system with “agent like” functionality in the form of different “modes” the user could select from a user interface (UI), this architecture can be seen on the left in figure 1. Three different modes were implemented, the first mode allowed the LLM to utilize the structures and algorithms within a graph database to trace UAM requirements. The second mode gave the LLM access to a vector search capable of providing relevant information from thousands of document pages related to UAM ConOps and requirements. The third mode served as a general assistant where users could enter open-ended questions and custom prompts to utilize the LLM for different use-cases. This system improved the process surrounding generating and analyzing information related to UAM requirements, however, the implementation provided a clunky user experience. Users were required to know what mode to select within the UI in advance before entering their question to the selected tool. Moreover, the different tools were isolated from each other, they lacked bidirectional links that would allow for tools to collaborate to generate better responses. Our team is working on a new architecture, seen on the right in the below figure, with the goal to address many of the UX shortcomings of our original system while improving the accuracy and depth of responses from the LLM. This new system will automatically select the appropriate tool to use based off the user’s question. Each tool will be capable of calling on any of the other tools available to the LLM, resulting in a collaborative pipeline where tools can pass data between other tools until enough data is received to generate an answer to the user’s question. Using a locally deployed, open-source, LLM, the NASA OCARI team, in collaboration with Collins Aerospace, will implement a prototype application that will bridge knowledge across multiple sources to assist System Engineers (SEs) with requirements discovery and tracing, research question and use case identification, and assumption validation. Such a system will also allow SEs to more easily, and intuitively, explore the AAM ecosystem, ultimately improving the efficiency and effectiveness of the SE's research and decision-making processes surrounding ConOps development and validation. In this session, our team will provide a video demonstration of our new prototype architecture in action. We will also present an overview of our prototype system architecture and talk about its advantages over traditional LLM deployments along with how those advantages can provide additional value to the field of System Engineering.

systems engineering↗

NASA NTRSDate not supplied

Requirement Discovery Using Embedded Knowledge Graph with ChatGPT

The field of Advanced Air Mobility (AAM) is witnessing a transformation with innovations such as electric aircraft and increasingly automated airspace operations. Within AAM, the Urban Air Mobility (UAM) concept focuses on providing air-taxi services in densely populated urban areas. This research introduces the utilization of Large Language Models (LLMs), such as OpenAI's GPT-4, to enhance the UAM Requirement discovery process. This study explores two distinct approaches to leverage LLMs in the context of UAM Requirement discovery. The first approach evaluates the LLM's ability to provide responses without relying on additional outside systems, such as a relational or graph database. Instead, a vector store provides relevant information to the LLM based on the user’s question, a process known as Retrieval Augmented Generation (RAG). The second approach integrates the LLM with a graph database. The LLM acts as an intermediary between the user and the graph database, translating user questions into cypher queries for the database and database responses into human-readable answers for the user. Our team implemented and tested both solutions to analyze requirements within a UAM dataset. This paper will talk about our approaches, implementations, and findings related to both approaches.

systems engineering↗

NASA NTRSDate not supplied

Requirement Discovery Using Embedded Knowledge Graph With ChatGPT

The field of Advanced Air Mobility (AAM) is witnessing a transformation with innovations such as electric aircraft and increasingly automated airspace operations. Within AAM, the Urban Air Mobility (UAM) con-cept focuses on providing air-taxi services in densely populated urban areas. This research introduces the utilization of Large Language Models (LLMs), such as OpenAI's GPT-4, to enhance the UAM Requirement discovery process. This study explores two distinct approaches to leverage LLMs in the context of UAM Requirement discovery. The first approach evaluates the LLM's ability to provide responses without relying on additional outside systems, such as a relational or graph database. Instead, a vector store provides relevant information to the LLM based on the user’s question, a process known as Retrieval Augmented Generation (RAG). The second approach integrates the LLM with a graph database. The LLM acts as an intermediary between the user and the graph database, translating user questions into cypher queries for the database and database responses into human-readable answers for the user. Our team implemented and tested both solutions to analyze require-ments within a UAM dataset. This paper will talk about our approaches, implementations, and findings related to both approaches.

systems engineering↗

NASA NTRSDate not supplied

Requirement Discovery Using Embedded Knowledge Graph With ChatGPT - Poster

The field of Advanced Air Mobility (AAM) is witnessing a transformation with innovations such as electric aircraft and increasingly automated airspace operations. Within AAM, the Urban Air Mobility (UAM) con-cept focuses on providing air-taxi services in densely populated urban areas. This research introduces the utilization of Large Language Models (LLMs), such as OpenAI's GPT-4, to enhance the UAM Requirement discovery process. This study explores two distinct approaches to leverage LLMs in the context of UAM Requirement discovery. The first approach evaluates the LLM's ability to provide responses without relying on additional outside systems, such as a relational or graph database. Instead, a vector store provides relevant information to the LLM based on the user’s question, a process known as Retrieval Augmented Generation (RAG). The second approach integrates the LLM with a graph database. The LLM acts as an intermediary between the user and the graph database, translating user questions into cypher queries for the database and database responses into human-readable answers for the user. Our team implemented and tested both solutions to analyze require-ments within a UAM dataset. This paper will talk about our approaches, implementations, and findings related to both approaches.

systems engineering↗

NASA NTRSDate not supplied

LLMs and GenAI Tools to Depict Contributions of Human Systems to Spaceflight Tasks Execution

Recent advancements in Artificial Intelligence and Machine Learning (AI/ML) technologies, particularly Large Language Models (LLMs) capable of sophisticated syntax analysis, offer substantial potential in automating complex processes, thereby saving time and human resources. This study explores the development of an LLM-driven model designed to analyze and categorize a diverse set of Mars mission tasks into 18 predefined Human System Task Categories (HSTCs) based on their textual descriptions. As part of developing the Crew Health and Performance – Probabilistic Risk Assessment (CHP-PRA projects Performance Risk Model (PRisM) proof-of-concept, we established a framework to project performance scores from small-scale tests onto a preliminary list of Mars tasks. The foundation of our model was a comprehensive spreadsheet populated by NASA experts and clinicians, which detailed each Mars task alongside binary indicators of HSTC involvement. This dataset enabled the initial application of supervised ML, training and testing on existing HSTC labels. The HSTCs were originally defined from a medical system perspective, focusing on task impairments due to deteriorated human health. To expand our model's scope to include categories impacting performance, we face the challenge of generating binary labels (0 or 1) for new categories without pre-existing data. We address this by employing Generative AI (GenAI) software to determine whether a given task involved a new category by asking, "Does task A involve using category B?" We validate our approach by comparing the GenAI's binary classifications with the expert-provided labels for existing HSTCs. Notably, we utilize Ollama [4], a locally hosted GenAI tool that does not require cloud access, thus safeguarding NASA's proprietary data from unauthorized exposure. This study demonstrates the feasibility of leveraging cutting-edge AI tools to advance research, paving the way for automation and rapid decision-making in space exploration.

Mona Matar↗