Search NASA⌕ Search

Engineering topics

Misty D Davies

Publications and source records attributed to Misty D Davies.

14 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2024

In-Time Safety Management for Part 139 Airports

Today’s airports are complex multi-faceted ecosystems. Currently, of the 517 certificated airports, 270 are required to use safety management systems (SMSs) to identify and mitigate known hazards and emergent risks and to voluntarily share safety data with commercial operators and tenants. Airports manage a wide variety of hazards. These traffic hubs have direct responsibilities, such as removing foreign object debris from runways and taxiways and configuring runways to help prevent against incursions and tail strikes during takeoff. To ensure safety in the future NAS, the National Academies recommended an In-time Aviation Safety Management System (IASMS). An IASMS will employ services, functions, and capabilities (SFCs) to identify and mitigate hazards that are proactively and predictively managed based on data analytics of detected anomalies, precursors, and trends. SFCs would scale with airport complexity and environmental conditions using increasingly automated systems to respond proactively to hazards and, by using integrated data sources and predictive safety analytical methods, discover new, never before seen risks.

IASMS↗
NASA NTRS2023

Research and Technology Challenges for Human Data Analysts in Future Safety Management Systems

Enabling new and novel concepts of operations for Advanced Air Mobility poses an important need to evolve current safety management systems (SMS) and is posited to be realized through advances in Machine Learning (ML) Data Sciences and Artificial Intelligence. The “In-time Aviation Safety Management System” (IASMS) concept of operations supports the need to evolve today’s SMS to become more tailorable, scalable, and interoperable in response to forecasted changes expected for the future airspace system. Key to IASMS is integration of proactive and predictive ML algorithms trained to provide “in time” detection and mitigation of hazards and emergent risks through new methods and novel data types. IASMS research and technology development includes human factors design considerations for these systems to include human-system teaming, innovations in human interfaces and management of complex digital data information, human-system interaction/model-based system engineering, and verification and validation for data assurance and trust.

Chad L Stephens↗
NASA NTRSDate not supplied

Augmenting Topic Finding in the NASA Aviation Safety Reporting System using Topic Modeling

Context: The NASA Aviation and Safety Reporting System (ASRS) provides various publications to the aviation community (including individual anonymous reports, Callback, Database Search Requests, Directline, and Alerting Messages). Key to these publications are the timely processing of new reports, which is currently done mostly manually by ASRS staff, and which the volume increases yearly. Aim: We investigate whether existing topic modelling techniques are suitable to ease some of the manual effort, and to enhance it with additional visual cues regarding the process of grouping, sense making and labeling incoming (and previous) reports. Method: We evaluate the applicability of WarpLDA topic modelling results combined with three visualization tools, the first two of which have been extended by us in this work for ASRS: Termite, TopicFlow, and LDAVis. Based on the identified limitations in these tools, we propose a methodology for improving them, and evaluate their outputs using ASRS as our test dataset. Results: The user interfaces of Termite, Topicflow and LDAVis were found insufficient for sense-making of the narratives. Moreover, concerns regarding the stability of results due to the inherent randomness of topic modelling, and the lack of a measurable approach for evaluation against the existing ASRS manual workflow were also noted. Conclusion: While many tools to topic modeling and visualization have been proposed, more work is necessary before they can be applied in practical situations to improve existing manual workflows. The methodology presented and applied in this work contribute towards this effort.

ASRS↗
NASA NTRSDate not supplied

A Survey Protocol to Assess Meaningfulness and Usefulness of Automated Topic Finding in the NASA Aviation Safety Reporting System

Context: The NASA Aviation Safety Reporting System (ASRS) is a voluntary confidential aviation safety reporting system. The ASRS receives reports from pilots, air traffic controllers, flight attendants and other involved in aviation operations. The reports are de-identified and coded by ASRS expert safety analysts and a short descriptive synopsis is written to describe the safety issue. The de-identified reports are then disseminated to the aviation community in a number of ways including entry into an online database, Safety Alert Bulletins and For Your Information Notices, and the CALLBACK newsletter. Key to these publications are the timely processing (de-identification, coding and summarization) of new reports, which is currently done by ASRS expert safety analysts. Thus, we believe topic modelling could decrease effort in ASRS, if topics are comprehensible. Aim: We propose a methodology to evaluate whether automated topic finding using topic modelling provides meaningful and useful topics. Method: We extend the total error survey methodology to evaluate user topic comprehension of machine learning outputs. To accomplish this we performed a literature review to identify existing methods and define a construct for topic comprehension, utilizing existing ASRS synopsis writing practices to more precisely define meaningfulness and usefulness. Results: A survey protocol was created that addresses the limitations of other survey protocols found in the literature review, which we found lacking in rationale and clear protocol definition. Conclusion: The surveying of user understanding in machine learning outputs presents challenges due to the explosion of parameters to control for and the lack of systematic approach presented in the literature. More reproducible work and survey protocols are needed in the literature and our work is one step towards that direction.

topic finding↗
NASA NTRSDate not supplied

Identifying Emerging Safety Threats Through Topic Modeling in the Aviation Safety Reporting System: A COVID-19 Study

The NASA Aviation Safety Reporting System (ASRS) is a voluntary, confidential aviation safety reporting system. The ASRS receives reports from pilots, air traffic controllers, flight attendants, and others involved in aviation operations. The reports are de-identified and coded by ASRS expert safety analysts, and a short descriptive synopsis is written to describe the safety issue. The de-identified reports are then disseminated to the aviation community in many ways, including via an online database, Safety Alert Bulletins, For Your Information Notices, and the CALLBACK newsletter. In this work, we consider whether we can improve the grouping, linking, and understanding of safety concerns through topic modeling. Specifically, we use topic modeling as a building block to identify emerging safety threats over time. This unsupervised approach, we argue, offers the flexibility to identify new emerging themes in this large dataset by constructing different timelines based on the content similarity of ASRS report narratives. This method's unsupervised nature improves upon related research, which is limited to pre-defined labels and therefore can not fully capture emerging safety threats. We apply our method to all ASRS reports in 2020 to assess if the generated timelines can highlight COVID-19 as it is emerging as a safety threat in incoming ASRS reports. We perform both a quantitative and qualitative evaluation of the automatically constructed timelines. The qualitative evaluation is performed by describing the evolution of top terms in the timelines, generated by our method, which we found explicitly convey the themes of COVID-19. Separately, we use a set of 1,213 COVID-19 reports from 2020 that were manually identified by ASRS analysts to quantitatively evaluate the COVID-19 reports distribution across the timelines. Our results have shown that COVID-19 emergence can be identified using the top terms that were generated by topic modeling. The top terms in topic modeling therefore can serve as a summary alternative to manually inspecting reports. Moreover, leveraging the manually identified COVID-19 reports, we found the manually identified timelines accounted for over 70% of the COVID-19 reports curated by the ASRS analysts, which demonstrates the potential of this approach for facilitating the understanding of safety concerns as they emerge and evolve. This method shows great potential to understand aerospace safety threats and other narrative- driven incident report databases.

ASRS↗
NASA NTRSDate not supplied

Human Interfaces and Management of Information (HIMI) Challenges for “In-time” Aviation Safety Management Systems (IASMS)

The envisioned transformation of the National Airspace System to integrate an In-time Aviation Safety Management System(IASMS)to assure safety in Advanced Air Mobility(AAM)brings unprecedented challenges to the design of human interfaces and management of safety information. Safety in design and operational safety assurance are critical factors for how humans will interact with increasingly autonomous systems. The IASMS Concept of Operations builds from traditional commercial operator safety management and scales in complexity to AAM. The transformative changes in future aviation systems pose potential new critical safety risks with novel types of aircraft and other vehicles having different performance capabilities, flying in increasingly complex airspace, and using adaptive contingencies to manage normal and non-normal operations. These changes compel development of new and emerging capabilities that enable innovative ways for humans to interact with data and manage information. In-creasing complexity of AAM corresponds with use of predictive modeling, data analytics, machine learning, and artificial intelligence to effectively address known hazards and emergent risks. The roles of humans will dynamically evolve in increments with this technological and operational evolution. The interfaces for how humans will interact with increasingly complex and assured systems designed to operate autonomously and how information will need to be presented are important challenges to be resolved.

Lawrence J Prinzel↗
NASA NTRSDate not supplied

Analysis of Input from Wildfire Incident Experts to Identify Key Risks and Hazards in Wildfire Emergency Response

The United States Department of Agriculture (USDA) describes wildland fires as, “a force of nature that can be nearly as impossible to prevent, and as difficult to control, as hurricanes, tornadoes and floods.” Existing challenges in managing wildland fires often put first responders’ lives at risk. The emergence of drones and their capabilities to supplement human efforts could alleviate some, if not all, of those risks that first responders face during wildfire management efforts. However, the process of adding drones to wildfire response has come with its own challenges as well. NASA’s System-Wide Safety Project is working towards overcoming these challenges to enable routine transfer of risk from responders to aviation assets. The concept of operations and model-based systems engineering (MBSE) effort for this shift is underway. To inform and to validate the MBSE effort, we delivered a questionnaire to wildland firefighting experts on the hazards they currently face. This questionnaire has given us insight and a better understanding of the challenges related to the use of drones from a first responder’s point of view. We are using this information to better address responders’ concerns, develop a safety management system, and eliminate the roadblocks that prevent the use of drones in wildfire management.

Wildfire↗
NASA NTRSDate not supplied

Human Factors Research Needs for In-Time Aviation Safety Management Systems (IASMS) Design: Enabling the NASA “Sky for All” Future Airspace Vision

Integrated safety management will be paramount for safely enabling the envisioned transformations of the future National Airspace System. Addressing the increasing need for advanced data analytics and fusion of aviation safety data, managed by human decision-makers, is essential for realizing the vision. The proposal, if accepted, will discuss safety management system challenges and how the concept of In-time Aviation Safety Management Systems addresses the need. It will also discuss human factors challenges involved in future integrated safety management, including trust, over-reliance, human-optimized data visualization, human-autonomy teaming, training, communication and dissemination of data, situation awareness, task load, and accountability.

Lawrence J Prinzel III↗
NASA NTRSDate not supplied

From BERTopic to SysML: Informing Model-Based Failure Analysis With Natural Language Processing for Complex Aerospace Systems

The development of emerging complex aerospace systems will require new approaches for capturing safety incident scenarios as early as possible in the design phase. However, for novel systems, relevant data available is limited. In this work, we propose a framework informing model-based mission assurance activities with historical incident reports, lessons learned, or other relevant engineering documents using natural language processing. In doing so, we investigate whether there is useful information in data sets that are relevant, if not identical, to the system under design and whether, through rigorous systems engineering practice, this information can be effectively leveraged through model-based failure analysis. In a worked case study, we apply state-of-the-art topic modeling techniques to two data sets, a mission relevant data set and a system relevant data set. The sets of topics are merged and interpreted to form a preliminary list of failure topics that can be used to inform the identification of off-nominal modes in the model-based failure modes and effects analysis development. Once data from the system in operation is available, it can be used to update the topics identified. By extracting information about likely failures from relevant historical data sets and utilizing model-based mission assurance to ensure relevance and rigor, unanticipated failures can be reduced, and projects can more effectively learn from past missions.

Failure Analysis↗
NASA NTRSDate not supplied

Kaona: Deep Searching and Curating Data from Aviation Safety Reporting Systems

Context: Several works in the literature have examined how safety narrative databases can be leveraged to share lessons learned. However, less attention has been given to augmenting existing processes for mining these safety reporting system databases. Aim: In this work, we introduce Kaona: An interface that weaves machine learning in existing aviation safety database mining activities. Method: We provide a use case of search, curation and newsletter writing to showcase how Kaona features build on existing processes and on its own to enhance information retrieval, curation and synthesis of narratives. Results: We created two instances of Kaona internally for evaluation, one using publicly available NASA’s ASRS narratives and another using publicly available C3RS narratives. Data ranged from 1998 to 2024. Conclusion: Our tool provides a new way to explore safety narratives, serving to re-imagine how text databases can benefit of novel information retrieval mechanisms in the era of large language models.

ASRS↗
NASA NTRSDate not supplied

Determining Optimal Asset Location for Rapid and Efficient Wildfire Suppression: A Simulation-Based Approach

The impact of wildfire incidents has been growing in recent years, posing a serious threat to communities at the urban-wildland interface. To address this problem, there have been growing calls to use UAVs to increase the capacity of responsible agencies to quickly and effectively suppress fires and to reduce risks associated with firefighting. One of the opportunities associated with UAVs is the ability to rapidly and autonomously operate from limited-access air bases where fires are expected to burn. This study provides an approach to determine where these air bases should be placed in order to most rapidly extinguish fires, given provided fuel distributions. This approach uses an integrated simulation of fire propagation and UAV-based suppression actions to determine how much of a given environment was burned over a range of scenarios. It then uses an optimization method to explore the space and determine the location with the least burned area. Results show the approach to efficiently and effectively provide optimal bases for single-base placements over a range of scenarios, though future work is required to adequately calibrate the model and study how it can be used in multiple-base placement problems.

Daniel Hulse↗