Results for “SEQUOIA” — Page 5

NASA NTRSDate not supplied

Semantic Search with Sentence-BERT for Design Information Retrieval

Managing and referencing design knowledge is a critical activity in the design process. However, reliably retrieving useful knowledge can be a frustrating experience for users of knowledge management systems due to inherent limitations of standard keyword-based searches. In this research, we consider the task of retrieving relevant lessons learned from the NASA Lessons Learned Information System (LLIS). To this end, we apply a state-of-the-art natural language processing (NLP) technique for information retrieval (IR): semantic search with sentence-BERT, which is a modification of a Bidirectional Encoder Representations from Transformers (BERT) model that uses siamese and triplet network architectures to obtain semantically meaningful sentence embeddings. While the pre-trained sBERT model performs well out-of-the-box, we further fine-tune the model on data from the LLIS so that it learns on design engineering-relevant vocabulary. We quantify the improvement in query results using both standard sBERT and fine-tuned sBERT over a keyword search. Our use case throughout the paper is to use queries related to specific requirements from a NASA project. Fine tuning the sBERT model on LLIS data yields a mean average precision (MAP) of 0.807 on queries based on information needs from a real NASA project. Results indicate that applying state-of-the-art natural language processing techniques, especially when finetuned using engineering data, to design information retrieval tasks shows significant promise in modernizing design knowledge management systems.

Hannah S. Walsh↗

NASA NTRSDate not supplied

What Went Wrong: A Survey of Wildfire UAS Mishaps through Named Entity Recognition

Increasingly, unmanned aircraft systems (UAS) are being applied to wildfire incidents for tasks such as mapping, aerial ignition, and delivery. As a result, incident reporting systems for wildfires are beginning to accumulate data related to UAS mishaps in wildfire response. In this research, we apply state-of-the-art natural language processing (NLP) techniques to develop a custom Named Entity Recognition (NER) model which extracts a Failure Modes and Effects Analysis (FMEA)-style survey of wildfire UAS mishaps reported in SAFECOM. The custom NER model is built by fine-tuning an existing (BERT) model, resulting in a generalizable NER model that can extract engineering relevant entities including failure modes, causes, effects, control processes, and recommendations from any failure-relevant text. Similar mishaps are clustered and reported as single rows within the FMEA. For each cluster, frequency, severity, and overall risk are computed. The methodology can be applied as part of a broader safety management system to track trends in mishaps and discover knowledge that can be utilized to improve safety outcomes and system performance.

Machine Learning↗

NASA NTRSDate not supplied

Machine Learning Enabled Quantitative Risk Assessment of Aerial Wildfire Response

Aerial wildfire operations are high risk and account for a large number of firefighter deaths. Increasing intensity of wildfires is driving a surge in aerial operations, while simultaneously there is growing interest in improving system safety and performance. In this work, wildfire aviation mishaps documented using the SAFECOM system are analyzed using a previously developed framework for hazard extraction and analysis of trends (HEAT). Hazards and specific failure modes are extracted from the narrative data in SAFECOM forms using natural language processing techniques. Metrics for each hazard are calculated, including frequency, rate, and severity. We examine whether these metrics change over time, and whether they are related to metadata, such as region and aircraft type. The results of the hazard analysis are presented in a risk matrix, identifying the highest and lowest risk hazards based on rate of occurrence and average severity. Results identify jumper operations hazards as high-risk, in addition to bucket drop failures, cargo let down failures, and severe weather as medium risk.

machine learning↗

NASA NTRSDate not supplied

Semantic Search with Sentence-BERT for Design Information Retrieval

Managing and referencing design knowledge is a critical activity in the design process. However, reliably retrieving useful knowledge can be a frustrating experience for users of knowledge management systems due to inherent limitations of standard keyword-based searches. In this research, we consider the task of retrieving relevant lessons learned from the NASA Lessons Learned Information System (LLIS). To this end, we apply a state-of-the-art natural language processing (NLP) technique for information retrieval: semantic search with sentence-BERT, which is a modification of a Bidirectional Encoder Representations from Transformers (BERT) model that uses siamese and triplet network architectures to obtain semantically meaningful sentence embeddings. While the pretrained sBERT model shows excellent out-of-the-box performance, we further fine-tune the model on data from the LLIS so that it learns on design engineering-relevant vocabulary. We quantify the improvement in query results using both standard sBERT and fine-tuned sBERT over the LLIS’s built-in keyword search. Additionally, we demonstrate a use case for the query system by searching for lessons learned relevant to specific requirements from a NASA project as part of a broader knowledge management and retrieval system. Results indicate that applying state-of-the-art natural language processing techniques, especially when fine-tuned using engineering data, to design information retrieval tasks shows significant promise in modernizing design knowledge management systems.

Hannah S. Walsh↗

NASA NTRSDate not supplied

Open Data Integration (ODIN): A Concurrent, Distributed Message-Based Architecture and Framework for Disaster Response

The Runtime for Airspace Concept Evaluation (RACE) is an open-source software architecture and framework to build configurable, highly concurrent and distributed message-based systems that offer scalable, low-latency performance on commodity hardware. RACE was used in commercial aviation applications to rapidly build systems that span several machines (including synchronized displays), interface existing hardware simulators and other live data feeds, and incorporate sophisticated visualization components such as NASA WorldWind. These RACE applications validated elements of the FAA’s System Wide Information Management (SWIM) Program, handling up to 1000 messages/sec from diverse sources (SFDPS, TFM-DATA, TAIS, ASDE-X, ITWS and local ADS) for 4,500 simultaneous flights tracked in the next-generation air transportation system’s digital backbone. We have since generalized RACE to support Open Data Integration (ODIN) applications outside aviation. Systems built with RACE/ODIN can be deployed in the field, on commodity hardware, and operate with limited or intermittent connectivity to the outside world. Our primary use case is a web-server with local/persistent data storage that runs within and only serves the stakeholder network (e.g. an incident command post). We are tailoring the RACE/ODIN system to support wildland fire management for the upcoming NASA Wildland Fire Safety Demonstration Series. RACE-ODIN is under consideration for application in the Scalable Traffic Management for Emergency Response Operations project, or STEReO, which aims to create a system that can be deployed during emergencies, to coordinate multiple elements of disaster response. Such data sources predominantly come from existing services on the internet (e.g. weather and satellite data, imported from so called "edge servers") but can also include dynamic (real-time) data from computer simulations and within the stakeholder network (such as aircraft and personnel tracking information). We will present the architecture and ODIN system demonstration incorporating local data from instrumented power-line towers, interpolated weather data and geospatial data from space-based platforms.

Joseph C Coughlan↗

NASA NTRSDate not supplied

What Went Wrong: A Survey of Wildfire UAS Mishaps through Named Entity Recognition

Increasingly, unmanned aircraft systems (UAS) are being applied to wildfire incidents for tasks such as mapping, aerial ignition, and delivery. As a result, aviation incident reporting systems for wildfires are beginning to accumulate data related to UAS mishaps in wildfire response. In this research, we apply state-of-the-art natural language processing (NLP) techniques to develop a custom Named Entity Recognition (NER) model which extracts entities relevant to safety analysts. The custom NER model is built by fine-tuning an existing Bidirectional Encoder Representations from Transformers (BERT) model, resulting in a generalizable NER model that can extract engineering relevant entities including failure modes, causes, effects, control processes, and recommendations from failure-relevant text. This model performs passably, with a weighted average f1 score of 0.33 across entity types, indicating more labeled training data is needed. Extracted entities are used to form a Failure Modes and Effects Analysis (FMEA)-style survey of wildfire UAS mishaps reported using the SAFECOM system. Similar mishaps are manually clustered and reported as single rows within an FMEA. Foreach cluster, we compute frequency, severity, and overall riskin accordance with FAA standards. This methodology can beapplied as part of a broader safety management system totrack trends in mishaps (e.g., likelihood, severity) and discoverknowledge (e.g., causes, effects) that can be utilized to improvesafety outcomes and system performance.

Machine Learning↗

NASA NTRS2023

Machine Learning Framework for Hazard Extraction and Analysis of Trends (HEAT) in Wildfire Response

This research proposes a natural language processing enabled risk analysis framework, named Hazard Extraction andAnalysis of Trends (HEAT), and applies the framework to the ICS-209-PLUS data set of wildfire incident responseforms. The HEAT framework produces safety- and risk- relevant analyses, consisting of: (1) a set of hazards extractedfrom text data, (2) a primary analysis using hazard-relevant metrics, such as rate and severity, to form an FMEA-styletable and risk matrix, (3) a time series analysis of metric trends, and (4) a secondary analysis examining potentialpredictors for hazards. Results from HEAT provide quantitative risk-relevant information for high-level hazards doc-umented in existing-state operations. Because of the generalizability of the steps and limited data requirements, HEATcan be applied to any dataset containing narrative text, thus providing a framework for data-driven machine learning-enabled quantitative risk analysis across a variety of domains. To demonstrate HEAT in a case study, we apply theframework to the ICS-209-PLUS dataset of wildland fire incident response forms. Hazards identified in wildfire re-sponse arise from environmental conditions, the mission, and the wildland urban interface. The resulting risk matrixidentifies evacuations as high-risk hazards, while all other identified hazards are medium or serious risk.

natural language processing↗

NASA NTRS2022

Supporting Hazard Analysis for Wildfire Response Using fmdtools and MIKA

The System Wide Safety (SWS) Safety Demonstrator (SD) Series drives development of an increasingly capable In-Time Aviation Safety Management System (IASMS) focusing on humanitarian applications, starting with wildfire response (SD-1). The goals of this report are to (1) provide an early hazard analysis and mitigation evaluation of wildfire response to support these efforts and (2) provide a demonstration of capabilities of the Fault Model Design Tools (fmdtools) and Manager for Intelligent Knowledge Access (MIKA) tools. fmdtools provides a modeling, simulation, and resiliency analysis framework in which a wildfire response model, the System Modeling and Analysis of Resiliency in Scalable Traffic Management for Emergency Response Operations (SMARt-STEReO), is built. MIKA is an intelligent knowledge manager with several capabilities, including assisting in hazard analysis by extracting and analyzing hazards from historical incident reports. The following topics are covered in the report: Understanding Wildfire Hazard Dynamics. We provide a description and simulated examples of how hazards occur in the SMARt-STEReO model of wildfire response and their effect on its outcome. This provides a common mental model and focuses the analysis presented in the remainder of the report. Wildfire Hazard Identification. MIKA identifies wildfire hazards from three relevant datasets: the ICS-209-PLUS, SAFECOM, and SAFENET. Hazards are manually organized into a taxonomy and MIKA analyzes each hazard’s effects, likelihood, severity, and risk. Evaluating Mitigation Strategies. The SMARt-STEReO wildfire response model built in fmdtools evaluates a subset of identified hazards. Specifically, we simulate the effect of communications faults and equipment faults on operator safety, the effect of changing winds and flammability, and a scenario with multiple ignition points and heavy smoke. Tool Limitations and Usage Considerations. We provide a discussion of appropriate tool use cases as well as limitations and considerations for usage. The tool findings are used to synthesize recommendations for wildfire response operations, which can be captured as part of an IASMS. Key recommendations are as follows: Hazards are identified from a broad spectrum of sources including aircraft subsystems, operational sources, and ground crew operations. Highest risk operational environment hazards identified are Evacuations. The highest risk manned aerial operations hazard categorized is Jumper Operations Mishap. Ground crew hazards that are highest risk are Burns, Cargo Operations Overhead, Dehydration, Entrapment, Falling Objects, Heart Attacks, Heat Exhaustion, Inadequate Training or Certification, Vehicle Breakdown, and Vehicle Collision. Modelled containment failures arise from a mismatch between the difficulty of the firefighting scenario and the capacity (e.g., speed, effectiveness, awareness) of the response. In firefighting scenarios where containment is possible (e.g., because the fire does not spread too quickly), these mismatches can occur because of a change in environmental conditions (e.g., wind, flammability, etc) or because of planning, equipment, or communications faults. Improvements to communications increase the capacity of the firefighting response by reducing the time needed to respond to the fire. While surveillance does not increase this capacity by itself, it increases operator safety by increasing state awareness, enabling firefighters to evade approaching fires. Increasing both has a synergistic effect. In general, these performance and resilience increases generalize over fault scenarios as well as unforeseen changes to circumstances (i.e., wind, aridity, etc.). However, these improvements need to be designed so as not to make the system prone to persistent large-scale communications outages, which can reduce performance.

Hazard analysis↗

NASA NTRS2022

NASA System-Wide Safety Wildland Firefighting Operations Workshop Report

On March 9-11, 2022, NASA’s System-Wide Safety Wildland Firefighting Operations Workshop engaged the broader wildland firefighting management ecosystem in a safety-oriented discussion via a virtual platform. This enabled a better understanding of how NASA and community expertise can be leveraged in the safe development of current and future firefighting systems and operations. The goals of the workshop were to: (1) identify and prioritize the top safety-oriented risks, gaps in capabilities, and emerging technologies to enhance wildland firefighting for both near-term and far-term concepts, with a specific focus on aviation operations and (2) engage the stakeholder community in defining emergent safety-oriented scope, roles, responsibilities, and procedures for agents undergoing increasingly complex wildland firefighting operations in information-rich, but uncertain environments. Workshop participants were solicited from wildland firefighting stakeholders across government, industry, and academia. All levels of government were engaged, as NASA sought attendees from federal, state, local, and tribal government agencies. Industry participants from traditional wildland firefighting domains such as data visualization and equipment manufacturers were invited, and corporate attendees from novel application domains such as aerial robotics and autonomous systems were present as well. The top three findings were as follows: (1) Enhancing situation awareness is a safety priority, especially in the use of aerial assets; (2) Timely access to information along with data fusion and integrated displays will enhance safety-critical decision-making both inside and outside aviation contexts; and (3) Tailorable standards and common operating pictures in the field will enhance inter-agency cooperation in the wildland firefighting lifecycle and enable the optimal use of limited resources such as aerial assets. The workshop helped inform NASA of the relevant safety-related wildland firefighting concerns and aided the broader ecosystem in understanding the potential safety-oriented role NASA might play in this community. Increased engagement with crucial governmental stakeholders (e.g., U.S. Forest Service, CAL FIRE, etc.) along with industry partners in cutting- edge information -centric domains is a fundamental next step. Additionally, the workshop findings will help define the first of a series of operationally challenging demonstrations, held in concert with strategic ecosystem partners, known as the Safety Demonstrator Series for NASA’s System-Wide Safety project. The first demonstration is set in the wildland firefighting application domain and will: (1) examine high risk operational scenarios to reduce their overall risk via services, functions or capabilities that act as risk mitigators (or transfer that risk to automated systems better able to tolerate it) and (2) explore novel tools and technologies that will enhance safety margins by enabling non-traditional or neoteric operational paradigms.

wildland firefighting↗

NASA NTRSDate not supplied

What Went Wrong: A Survey of Wildfire UAS Mishaps through Named Entity Recognition

Increasingly, unmanned aircraft systems (UAS) are being applied to wildfire incidents for tasks such as mapping, aerial ignition, and delivery. As a result, incident reporting systems for wildfires are beginning to accumulate data related to UAS mishaps in wildfire response. In this research, we apply state-of-the-art natural language processing (NLP) techniques to develop a custom Named Entity Recognition (NER) model which extracts a Failure Modes and Effects Analysis (FMEA)-style survey of wildfire UAS mishaps reported in SAFECOM. The custom NER model is built by fine-tuning an existing (BERT) model, resulting in a generalizable NER model that can extract engineering relevant entities including failure modes, causes, effects, control processes, and recommendations from any failure-relevant text. Similar mishaps are clustered and reported as single rows within the FMEA. For each cluster, frequency, severity, and overall risk are computed. The methodology can be applied as part of a broader safety management system to track trends in mishaps and discover knowledge that can be utilized to improve safety outcomes and system performance.

Machine Learning↗

NASA NTRS2022

Natural Language Processing Techniques for Intelligent Knowledge Management of Safety Reports

Safety, failure, and incident reports are common artifacts across various domains, including aviation and wildfire response. These reports are often mandatory to submit, resulting in the culmination of large repositories of text-based documents. Simultaneously, these reports and corresponding repositories are often only manually analyzed and queried by users via out-of-date search engines. As a consequence, we have been developing the Manager for Intelligent Knowledge Access (MIKA) toolkit, which uses natural language processing to improve information access and reuse. In this presentation, we discuss natural language processing techniques for knowledge discovery and apply these methods to a repository of aerial wildfire mishap reports. Two methods are used for knowledge discovery: topic modeling and named-entity recognition. We use topic modeling to identify hazards and perform a trend analysis to produce a data-driven risk matrix. A custom named-entity recognition model, build from fine tuning a pre-trained language model, is used to identify failure modes, failure causes, failure effects, control processes, and recommendations to aid in failure modes and effects analysis (FMEA). Throughout the presentation, we discuss and apply natural language processing techniques to better leverage the vast amount of information contained in report repositories.

Machine learning↗

NASA NTRSDate not supplied

MIKA: Manager for Intelligent Knowledge Access Toolkit for Engineering Knowledge Discovery and Information Retrieval

Repositories of safety reports are often underutilized and only analyzed manually by trained experts, despite safety management systems requiring reports. These collections of documents contain a wealth of information from past projects and operations that could improve system safety and design. Advances in natural language processing techniques have improved information extraction and retrieval in consumer technology, biomedicine, and finance, for instance, but have not been applied to engineering documents on the same scale. To this end, the Manager for Intelligent Knowledge Access (MIKA) open-source toolkit has been developed for rapid knowledge discovery and information retrieval in safety engineering applications. The MIKA toolkit uses state-of-the-art natural language processing algorithms and allows a user to apply these methods to their own dataset. This paper describes the MIKA toolkit and its two primary capabilities, knowledge discovery and information retrieval, and demonstrates the toolkit via a case study on National Transportation Safety Board (NTSB) reports.

Machine Learning↗

NASA NTRSDate not supplied

Open Data Integration (ODIN): A Concurrent, Distributed Message-Based Architecture and Framework for Disaster Response

The Runtime for Airspace Concept Evaluation (RACE) is an open-source software architecture and framework to build configurable, highly concurrent and distributed message-based systems that offer scalable, low-latency performance on commodity hardware. RACE was used in commercial aviation applications to rapidly build systems that span several machines (including synchronized displays), interface existing hardware simulators and other live data feeds, and incorporate sophisticated visualization components such as NASA WorldWind. These RACE applications validated elements of the FAA’s System Wide Information Management (SWIM) Program, handling up to 1000 messages/sec from diverse sources (SFDPS, TFM-DATA, TAIS, ASDE-X, ITWS and local ADS) for 4,500 simultaneous flights tracked in the next-generation air transportation system’s digital backbone. We have since generalized RACE to support Open Data Integration (ODIN) applications outside aviation. Systems built with RACE/ODIN can be deployed in the field, on commodity hardware, and operate with limited or intermittent connectivity to the outside world. Our primary use case is a web-server with local/persistent data storage that runs within and only serves the stakeholder network (e.g. an incident command post). We are tailoring the RACE/ODIN system to support wildland fire management for the upcoming NASA Wildland Fire Safety Demonstration Series. RACE-ODIN is under consideration for application in the Scalable Traffic Management for Emergency Response Operations project, or STEReO, which aims to create a system that can be deployed during emergencies, to coordinate multiple elements of disaster response. Such data sources predominantly come from existing services on the internet (e.g. weather and satellite data, imported from so called "edge servers") but can also include dynamic (real-time) data from computer simulations and within the stakeholder network (such as aircraft and personnel tracking information). We will present the architecture and ODIN system demonstration incorporating local data from instrumented power-line towers, interpolated weather data and geospatial data from space-based platforms.

Guillaume P Brat↗

NASA NTRSDate not supplied

MIKA: Manager for Intelligent Knowledge Access Toolkit for Engineering Knowledge Discovery and Information Retrieval

Repositories of safety reports are often underutilized and only analyzed manually by trained experts, despite safety management systems requiring reports. These collections of documents contain a wealth of information from past projects and operations that could improve system safety and design. Advances in natural language processing techniques have improved information extraction and retrieval in consumer technology, biomedicine, and finance, for instance, but have not been applied to engineering documents on the same scale. To this end, the Manager for Intelligent Knowledge Access (MIKA) open-source toolkit has been developed for rapid knowledge discovery and information retrieval in safety engineering applications. The MIKA toolkit uses state-of-the-art natural language processing algorithms and allows a user to apply these methods to their own dataset. This paper describes the MIKA toolkit and its two primary capabilities, knowledge discovery and information retrieval, and demonstrates the toolkit via a case study on National Transportation Safety Board (NTSB) reports.

Systems Engineering↗

NASA NTRSDate not supplied

SafeAeroBERT: Towards a Safety-Informed Aerospace-Specific Language Model

As aviation systems continue to operate with high traffic, large amounts of documents containing safety-relevant data continue to be generated via reporting systems such as the Aviation Safety Reporting System (ASRS). Advanced natural language processing techniques, specifically pre-trained language models, have shown great success in domain-specific applications; however, the text in aviation safety reports is inundated with jargon and thus not fully utilized by general pre-trained models. In this research, we work towards developing a safety-informed aerospace-specific language model by pre-training a Bidirectional Encoder Representations from Transformer (BERT) model on reports from the Aviation Safety Reporting System and the National Transportation Safety Board. The resulting model, called SafeAeroBERT, is fine-tuned for the specific task of document classification, and can be further tuned for named-entity recognition, relation detection, information retrieval, and summarization. Results from the classification task are compared between SafeAeroBERT, the base BERT, and SciBERT models and show SafeAeroBERT outperforms the general BERT and SciBERT on classifying reports about weather and procedure. SafeAeroBERT can be used on custom tasks, not limited to document classification, and is intended to aid an intelligent knowledge manager for safety report repositories.

Aviation↗

NASA NTRS2023

SafeAeroBERT: Towards a Safety-Informed Aerospace-Specific Language Model

As aviation systems continue to operate with high traffic, large amounts of documents containing safety-relevant data continue to be generated via reporting systems such as the ASRS. Advanced natural language processing techniques, specifically pre-trained language models, have shown great success in domain-specific applications; however, the text in aviation safety reports is inundated with jargon and thus not fully utilized by general pre-trained models. In this research, we work towards developing a safety-informed aerospace-specific language model by pre-training a Bidirectional Encoder Representations from Transformer (BERT) model on reports from the Aviation Safety Reporting System and the National Transportation Safety Board. The resulting model, called SafeAeroBERT, is fine-tuned for the specific task of document classification, and can be further tuned for named-entity recognition, relation detection, information retrieval, and summarization. Results from the classification task are compared between SafeAeroBERT, the base BERT, and SciBERT models and show SafeAeroBERT outperforms the general BERT and SciBERT on classifying reports about human factors, aircraft, and procedure. SafeAeroBERT can be used on custom tasks, not limited to document classification, and is intended to aid an intelligent knowledge manager for safety report repositories.

Aviation↗

NASA NTRSDate not supplied

Evaluating Faulty State Occurrence in Wildfire UAS Missions Using Markov Chains

As autonomous technology advances, unmanned aircraft systems are increasingly integrated into emergency response missions, such as wildfire response. These systems must be be safe with less risk than non-autonomous counter parts, yet quantifying the risk associated with present-day and future systems conventionally relies solely on expert opinion and little data. Instead, combining narrative mishap reports with probabilistic analysis can provide a method for evolutionary and timely risk analysis. In this paper, we present a framework for a data-driven probabilistic risk assessment style analysis, where hazard events and rates originate from documented UAS mishaps. The framework is applied to a UAS mapping mission in wildfire response, including a fault tree analysis, event tree analysis, and probabilistic analysis using Markov Chains. The analysis provides an enumeration of hazards in the system, hazard events that can lead to faults, the probability of a mission experiencing any fault, the probability of experiencing a specific fault, and the expected time spent until faulty states occur in present-day operations.

risk analysis↗

NASA NTRSDate not supplied

Evaluating Faulty State Occurrence in Wildfire UAS Missions Using Markov Chains

As autonomous technology advances, unmanned aircraft systems are increasingly integrated into emergency response missions, such as wildfire response. These systems must be be safe with less risk than non-autonomous counter parts, yet quantifying the risk associated with present-day and future systems conventionally relies solely on expert opinion and little data. Instead, combining narrative mishap reports with probabilistic analysis can provide a method for evolutionary and timely risk analysis. In this paper, we present a framework for a data-driven probabilistic risk assessment style analysis, where hazard events and rates originate from documented UAS mishaps. The framework is applied to a UAS mapping mission in wildfire response, including a fault tree analysis, event tree analysis, and probabilistic analysis using Markov Chains. The analysis provides an enumeration of hazards in the system, hazard events that can lead to faults, the probability of a mission experiencing any fault, the probability of experiencing a specific fault, and the expected time spent until faulty states occur in present-day operations.

risk analysis↗