Search NASA⌕ Search

SEARCH · Search NASA

Results for “Pre-trained models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2022

Microstructure Segmentation with Deep Learning Encoders Pre-Trained on a Large Microscopy Dataset

This study examined the improvement of microscopy segmentation accuracy by transfer learning from a large dataset of microscopy images called MicroNet. Many neural network encoder architectures, including VGG, Inception, and ResNet, were trained on over 100,000 labelled microscopy images from 54 classes. These pre-trained encoders were then embedded into multiple segmentation architectures including U-Net and DeepLabV3+ to evaluate segmentation performance on newly created benchmark microscopy datasets. Compared to ImageNet pre-training, models pre-trained on MicroNet generalized better to out-of-distribution micrographs taken under different imaging and sample conditions and were more accurate with less training data. When training with only a single Ni-superalloy image, pre-training on MicroNet produced a 72.2 percent reduction in relative segmentation error. These results suggest that transfer learning from large in-domain datasets generate models with learned feature representations that are more useful for downstream tasks and will likely improve any microscopy image analysis technique that can leverage pre-trained encoders.

machine learning↗
NASA NTRS2022

Microstructure Segmentation With Deep Learning Encoders Pre-Trained on a Large Microscopy Dataset

This study examined the improvement of microscopy segmentation intersection over union accuracy by transfer learning from a large dataset of microscopy images called MicroNet. Many neural network encoder architectures were trained on over 100,000 labeled microscopy images from 54 material classes. These pre-trained encoders were then embedded into multiple segmentation architectures including UNet and DeepLabV3+ to evaluate segmentation performance on created benchmark microscopy datasets. Compared to ImageNet pre-training, models pre-trained on MicroNet generalized better to out-of-distribution micrographs taken under different imaging and sample conditions and were more accurate with less training data. When training with only a single Ni-superalloy image, pre-training on MicroNet produced a 72.2% reduction in relative intersection over union error. These results suggest that transfer learning from large in-domain datasets generate models with learned feature representations that are more useful for downstream tasks and will likely improve any microscopy image analysis technique that can leverage pre-trained encoders.

machine learning↗
NASA NTRS2022

Neurosymbolic Hybrid Approach to Driver Collision Warning

There are two main algorithmic approaches to autonomous driving systems: (1) An end-to-end system in which a single deep neural network learns to map sensory input directly into appropriate warning and driving responses. (2) A mediated hybrid recognition system in which a system is created by combining independent modules that detect each semantic feature. While some researchers believe that deep learning can solve any problem, others believe that a more engineered and symbolic approach is needed to cope with complex environments with less data. Deep learning alone has achieved state-of-the-art results in many areas, from complex gameplay to predicting protein structures. In particular, in image classification and recognition, deep learning models have achieved accuracies as high as humans. But sometimes it can be very difficult to debug if the deep learning model doesn't work. Deep learning models can be vulnerable and are very sensitive to changes in data distribution. Generalization can be problematic. It's usually hard to prove why it works or doesn't. Deep learning models can also be vulnerable to adversarial attacks. Here, we combine deep learning-based object recognition and tracking with an adaptive neurosymbolic network agent, called the Non-Axiomatic Reasoning System (NARS), that can adapt to its environment by building concepts based on perceptual sequences. We achieved an improved intersection-over-union (IOU) object recognition performance of 0.65 in the adaptive retraining model compared to IOU 0.31 in the COCO data pre-trained model. We improved the object detection limits using RADAR sensors in a simulated environment, and demonstrated the weaving car detection capability by combining deep learning-based object detection and tracking with a neurosymbolic model.

Wang, Pei↗
NASA NTRS2022

Natural Language Processing Techniques for Intelligent Knowledge Management of Safety Reports

Safety, failure, and incident reports are common artifacts across various domains, including aviation and wildfire response. These reports are often mandatory to submit, resulting in the culmination of large repositories of text-based documents. Simultaneously, these reports and corresponding repositories are often only manually analyzed and queried by users via out-of-date search engines. As a consequence, we have been developing the Manager for Intelligent Knowledge Access (MIKA) toolkit, which uses natural language processing to improve information access and reuse. In this presentation, we discuss natural language processing techniques for knowledge discovery and apply these methods to a repository of aerial wildfire mishap reports. Two methods are used for knowledge discovery: topic modeling and named-entity recognition. We use topic modeling to identify hazards and perform a trend analysis to produce a data-driven risk matrix. A custom named-entity recognition model, build from fine tuning a pre-trained language model, is used to identify failure modes, failure causes, failure effects, control processes, and recommendations to aid in failure modes and effects analysis (FMEA). Throughout the presentation, we discuss and apply natural language processing techniques to better leverage the vast amount of information contained in report repositories.

Machine learning↗
NASA NTRSDate not supplied

Semantic Search with Sentence-BERT for Design Information Retrieval

Managing and referencing design knowledge is a critical activity in the design process. However, reliably retrieving useful knowledge can be a frustrating experience for users of knowledge management systems due to inherent limitations of standard keyword-based searches. In this research, we consider the task of retrieving relevant lessons learned from the NASA Lessons Learned Information System (LLIS). To this end, we apply a state-of-the-art natural language processing (NLP) technique for information retrieval (IR): semantic search with sentence-BERT, which is a modification of a Bidirectional Encoder Representations from Transformers (BERT) model that uses siamese and triplet network architectures to obtain semantically meaningful sentence embeddings. While the pre-trained sBERT model performs well out-of-the-box, we further fine-tune the model on data from the LLIS so that it learns on design engineering-relevant vocabulary. We quantify the improvement in query results using both standard sBERT and fine-tuned sBERT over a keyword search. Our use case throughout the paper is to use queries related to specific requirements from a NASA project. Fine tuning the sBERT model on LLIS data yields a mean average precision (MAP) of 0.807 on queries based on information needs from a real NASA project. Results indicate that applying state-of-the-art natural language processing techniques, especially when finetuned using engineering data, to design information retrieval tasks shows significant promise in modernizing design knowledge management systems.

Hannah S. Walsh↗
NASA NTRSDate not supplied

2022 Spring Internship Exit Presentation

As efforts of the National Aeronautics and Space Administration (NASA) and the Federal Aviation Administration (FAA) continue to digitize the air traffic management (ATM) domain, there is countless times of need for downstream natural language processing (NLP) tasks such as named entity recognition, text summarization, classification, and more. Although there are a plethora of open-sourced pre-trained transformer models in the NLP field such as BERT, RoBERTa, XLNet, and GPT-3, these models are trained on general corpora and perform poorly on domain-specific terminology and phraseology seen in ATM documents such as Notice to Airmen (NOTAMs) and Letters of Agreement (LoA). Our proposed research objective will be to first gather a large corpus of air traffic management related documents, orders, notices, books, technical papers, conference papers, articles, and other miscellaneous sources of text data from the FAA, NASA, and accredited conference and publication societies. After gathering this data, many steps will have to be taken to collate and preprocess the data into a format understandable by our test transformer models. Thirdly, we will set up training pipelines to train the RoBERTa model on its unsupervised training task masked language modelling (MLM) using resources provided by the NASA Advanced Supercomputing (NAS) facilities. Finally, these fine-tuned transformer models will be evaluated on their performance on down-stream NLP tasks as mentioned above, to show whether they will be effective when working with ATM related data or not. Once complete, this model could be made open-sourced on the HuggingFace website, where the rest of the ATM community can access and utilize this tool.

NLP↗
NASA NTRS2018

Enriching the Twitter Stream Increasing Data Mining Yield and Quality Using Machine Learning

Social media data streams are important sources of real-time and historical global information for science applications. At the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), we are exploring the Twitter data stream for its potential in augmenting the validation program of NASA Earth science missions, specifically the Global Precipitation Measurement (GPM) mission. We have implemented a tweet processing infrastructure that outputs classified precipitation tweets. Inputs are "passive" tweets, along with a smaller number of tweets from "active" participants, i.e., those knowingly contributing to our effort. The "active" tweets, presumably of higher quality, enrich the Twitter stream. "Active" sources include data scraped from other social media (e.g., public Facebook posts) and data from existing crowdsourcing programs (e.g., mPING reports). In addition, there is likely relevant precipitation information in images and documents that are the end points of links often included in tweets. Information derived from these "active" sources could then be tweeted into the Twitter stream, thus enriching its quality. The objective of our current work is to mine these tweet­ linked images and documents, using neural networks, to increase the information content and quality related to precipitation. For images, we classified them as either precipitation-related or not. For training and validation, we used images obtained via the Google custom search API. We created two models: (1) by training a simple Convolutional Neural Network and (2) by using transfer learning principles to adapt a pre-trained object recognition model. For documents, both those linked to tweets and the tweet contents, we trained Hierarchical Attention Networks to determine precipitation occurrence, type, and intensity. For training and validation, we used a keyword-filtered tweet data set labelled with ground truth data from Dark Sky (an API to retrieve weather-related labels) and the National Severe Storms Laboratory's Multi­ Radar/Multi-Sensor (MRMS) system. Our results demonstrated the efficacy of our machine learning approaches for enriching the Twitter stream, to derive information potentially useful for validation of earth science satellite data.

Albayrak, Arif↗
NASA NTRS2021

Highland Lakes Water Resources: Using NASA Earth Observations to Improve Detection Systems for Harmful Algal Events in the Highland Lakes in Central Texas

Beginning in 2019, harmful algal events in Austin, Texas, caused canine deaths in the Lady Bird Lake and Lake Travis reservoirs. These reservoirs are part of the larger Highland Lakes chain, managed by the Lower Colorado River Authority (LCRA) and the City of Austin Department of Watershed Protection (CoA DWP), which fulfill municipal, commercial, and agricultural water demands. Given the recent increase in favorable algal event conditions in central Texas, the LCRA and CoA DWP partnered with NASA DEVELOP to improve algal event early-warning systems through the application of remote sensing and machine learning. An Earth observation-based algal monitoring system will assist the responsible agencies in predicting algal conditions and communicating hazards to the public. The NASA DEVELOP team utilized Landsat 8 Operational Land Imager (OLI) and Sentinel-2 Multispectral Instrument (MSI) data to produce products including chlorophyll-a concentrations, cyanobacteria detections, turbidity, and water surface temperature. Chlorophyll-a concentrations were retrieved with a pre-trained machine learning model (mixture density network) and spectral indices, while the other products were derived from spectral indices. In situ field data were used to validate and quantify uncertainties for each product. The validations show strong correlations for chlorophyll-a and water surface temperature. Time series analyses of chlorophyll-a concentrations show peaks in the severe drought years (2015 and 2016). This project's resulting products enable monitoring of environmental proxies relevant to algal event presence in the Highland Lakes chain and will ultimately support water management, decision making, and risk communication.

Kaitlynn Hietpas↗
NASA NTRS2020

Shedding Light on Microbial Dark Matter with A Universal Language of Life

The majority of microbial genomes have yet to be cultured, and most proteins predicted from microbial genomes or sequenced from the environment cannot be functionally annotated. As a result, current computational approaches to describe microbial systems rely on incomplete reference databases that cannot adequately capture the full functional diversity of the microbial tree of life, limiting our ability to model high-level features of biological sequences. The scientific community needs a means to capture the functionally and evolutionarily relevant features underlying biology, independent of our incomplete reference databases. Such a model can form the basis for transfer learning tasks, enabling downstream applications in environmental microbiology, medicine, and bioengineering. Here we present LookingGlass, a deep learning model capturing a “universal language of life”. LookingGlass encodes contextually-aware, functionally and evolutionarily relevant representations of short DNA reads, distinguishing reads of disparate function, homology, and environmental origin. We demonstrate the ability of LookingGlass to be fine-tuned to perform a range of diverse tasks: to identify novel oxidoreductases, to predict enzyme optimal temperature, and to recognize the reading frames of DNA sequence fragments. LookingGlass is the first contextually-aware, general purpose pre-trained “biological language” representation model for short-read DNA sequences. LookingGlass enables functionally relevant representations of otherwise unknown and unannotated sequences, shedding light on the microbial dark matter that dominates life on Earth.

A Hoarfrost↗
NASA NTRS2003

Integration of Online Parameter Identification and Neural Network for In-Flight Adaptive Control

An indirect adaptive system has been constructed for robust control of an aircraft with uncertain aerodynamic characteristics. This system consists of a multilayer perceptron pre-trained neural network, online stability and control derivative identification, a dynamic cell structure online learning neural network, and a model following control system based on the stochastic optimal feedforward and feedback technique. The pre-trained neural network and model following control system have been flight-tested, but the online parameter identification and online learning neural network are new additions used for in-flight adaptation of the control system model. A description of the modification and integration of these two stand-alone software packages into the complete system in preparation for initial flight tests is presented. Open-loop results using both simulation and flight data, as well as closed-loop performance of the complete system in a nonlinear, six-degree-of-freedom, flight validated simulation, are analyzed. Results show that this online learning system, in contrast to the nonlearning system, has the ability to adapt to changes in aerodynamic characteristics in a real-time, closed-loop, piloted simulation, resulting in improved flying qualities.

Hageman, Jacob↗
NASA NTRS2003

Integration of Online Parameter Identification and Neural Network for In-Flight Adaptive Control

An indirect adaptive system has been constructed for robust control of an aircraft with uncertain aerodynamic characteristics. This system consists of a multilayer perceptron pre-trained neural network, online stability and control derivative identification, a dynamic cell structure online learning neural network, and a model following control system based on the stochastic optimal feedforward and feedback technique. The pre-trained neural network and model following control system have been flight-tested, but the online parameter identification and online learning neural network are new additions used for in-flight adaptation of the control system model. A description of the modification and integration of these two stand-alone software packages into the complete system in preparation for initial flight tests is presented. Open-loop results using both simulation and flight data, as well as closed-loop performance of the complete system in a nonlinear, six-degree-of-freedom, flight validated simulation, are analyzed. Results show that this online learning system, in contrast to the nonlearning system, has the ability to adapt to changes in aerodynamic characteristics in a real-time, closed-loop, piloted simulation, resulting in improved flying qualities.

Hageman, Jacob J.↗
NASA NTRS2004

Intelligent Control for the BEES Flyer

This paper describes the effort to provide a preliminary capability analysis and a neural network based adaptive flight control system for the JPL-led BEES aircraft project. The BEES flyer was envisioned to be a small, autonomous platform with sensing and control systems mimicking those of biological systems for the purpose of scientific exploration on the surface of Mars. The platform is physically tightly constrained by the necessity of efficient packing within rockets for the trip to Mars. Given the physical constraints, the system is not an ideal configuration for aerodynamics or stability and control. The objectives of this effort are to evaluate the aerodynamics characteristics of the existing design, to make recommendaaons as to potential improvements and to provide a control system that stabilizes the existing aircraft for nominal flight and damaged conditions. Towards this several questions are raised and analyses are presented to arrive at answers to some of the questions raised. CART3D, a high-fidelity inviscid analysis package for conceptual and preliminary aerodynamic design, was used to compute a parametric set of solutions over the expected flight domain. Stability and control derivatives were extracted from the database and integrated with the neural flight control system. The Integrated Vehicle Modeling Environment (IVME) was also used for estimating aircraft geometric, inertial, and aerodynamic characteristics. A generic neural flight control system is used to provide adaptive control without the requirement for extensive gain scheduling or explicit system identification. The neural flight control system uses reference models to specify desired handling qualities in the roll, pitch, and yaw axes, and incorporates both pre-trained and on-line learning neural networks in the inverse model portion of the controller. Results are presented for the BEES aircraft in the subsonic regime for terrestrial and Martian environments.

Krishnakumar, K.↗
NASA NTRS2022

Mars Terrain Segmentation with Less Labels

Planetary rover systems need to perform terrain segmentation to identify drivable areas as well as identify specific types of soil for sample collection. The latest Martian terrain segmentation methods rely on supervised learning which is very data hungry and difficult to train where only a small number of labeled samples are available. Moreover, the semantic classes are defined differently for different applications (e.g., rover traversal vs. geological) and as a result the network has to be trained from scratch each time, which is an inefficient use of resources. This research proposes a semi-supervised learning framework for Mars terrain segmentation where a deep segmentation network trained in an unsupervised manner on unlabeled images is transferred to the task of terrain segmentation trained on few labeled images. The network incorporates a backbone module which is trained using a contrastive loss function and an output atrous convolution module which is trained using a pixel-wise cross-entropy loss function. Evaluation results using the metric of segmentation accuracy show that the proposed method with contrastive pre-training outperforms plain supervised learning by 2%-10%. Moreover, the proposed model is able to achieve a segmentation accuracy of 91.1% using only 161 training images (1% of the original dataset) compared to 81.9% with plain supervised learning.

Wilson, Brian D↗
NASA NTRS1997

Evaluation of a Neural Adaptive Flight Controller

The objective of this paper is to present results from the evaluation of a direct adaptive tracking controller. The control architecture employs both pre-trained and an on-line neural networks to represent the non-linear aircraft dynamics in the model inversion portion of the controller. The aircraft model used for this evaluation is representative of the F-15 Advanced Control Technology for Integrated Vehicles (ACTIVE) aircraft. The controller was evaluated for three cases: (1) nominal conditions; (2) loss of control power; and (3) loss of control power in the presence of atmospheric turbulence. The results were compared with the existing F-15 ACTIVE conventional mode controller in all cases. The results indicate extremely desirable airframe stabilization characteristics for case (1) that do not degrade significantly for case (2) or (3) as does the conventional mode controller. It was concluded that this controller exhibits both stable and robust adaptive characteristics when subjected to mild and extreme loss of control power conditions. Integration of this neural adaptive flight controller into the full non-linear six degree-of-freedom F-15 ACTIVE simulation is recommended for evaluation in a real-time high fidelity piloted simulation environment.

Totah, Joseph J.↗
NASA NTRS2003

Flight Test of an Intelligent Flight-Control System

The F-15 Advanced Controls Technology for Integrated Vehicles (ACTIVE) airplane (see figure) was the test bed for a flight test of an intelligent flight control system (IFCS). This IFCS utilizes a neural network to determine critical stability and control derivatives for a control law, the real-time gains of which are computed by an algorithm that solves the Riccati equation. These derivatives are also used to identify the parameters of a dynamic model of the airplane. The model is used in a model-following portion of the control law, in order to provide specific vehicle handling characteristics. The flight test of the IFCS marks the initiation of the Intelligent Flight Control System Advanced Concept Program (IFCS ACP), which is a collaboration between NASA and Boeing Phantom Works. The goals of the IFCS ACP are to (1) develop the concept of a flight-control system that uses neural-network technology to identify aircraft characteristics to provide optimal aircraft performance, (2) develop a self-training neural network to update estimates of aircraft properties in flight, and (3) demonstrate the aforementioned concepts on the F-15 ACTIVE airplane in flight. The activities of the initial IFCS ACP were divided into three Phases, each devoted to the attainment of a different objective. The objective of Phase I was to develop a pre-trained neural network to store and recall the wind-tunnel-based stability and control derivatives of the vehicle. The objective of Phase II was to develop a neural network that can learn how to adjust the stability and control derivatives to account for failures or modeling deficiencies. The objective of Phase III was to develop a flight control system that uses the neural network outputs as a basis for controlling the aircraft. The flight test of the IFCS was performed in stages. In the first stage, the Phase I version of the pre-trained neural network was flown in a passive mode. The neural network software was running using flight data inputs with the outputs provided to instrumentation only. The IFCS was not used to control the airplane. In another stage of the flight test, the Phase I pre-trained neural network was integrated into a Phase III version of the flight control system. The Phase I pretrained neural network provided realtime stability and control derivatives to a Phase III controller that was based on a stochastic optimal feedforward and feedback technique (SOFFT). This combined Phase I/III system was operated together with the research flight-control system (RFCS) of the F-15 ACTIVE during the flight test. The RFCS enables the pilot to switch quickly from the experimental- research flight mode back to the safe conventional mode. These initial IFCS ACP flight tests were completed in April 1999. The Phase I/III flight test milestone was to demonstrate, across a range of subsonic and supersonic flight conditions, that the pre-trained neural network could be used to supply real-time aerodynamic stability and control derivatives to the closed-loop optimal SOFFT flight controller. Additional objectives attained in the flight test included (1) flight qualification of a neural-network-based control system; (2) the use of a combined neural-network/closed-loop optimal flight-control system to obtain level-one handling qualities; and (3) demonstration, through variation of control gains, that different handling qualities can be achieved by setting new target parameters. In addition, data for the Phase-II (on-line-learning) neural network were collected, during the use of stacked-frequency- sweep excitation, for post-flight analysis. Initial analysis of these data showed the potential for future flight tests that will incorporate the real-time identification and on-line learning aspects of the IFCS.

Davidson, Ron↗
NASA NTRSDate not supplied

The Application of Remote Sensing and Machine Learning to Improve Early Warning Systems for Harmful Algal Events in the Highland Lake Chain, TX

Beginning in 2019, harmful algal events have caused canine deaths in both Lady Bird Lake and Lake Travis located near Austin, Texas. These two reservoirs are part of the larger Highland Lakes chain, managed by the City of Austin Department of Watershed Protection (COA DWP) and the Lower Colorado River Authority (LCRA), which fulfill municipal, commercial, and agricultural water demands. Given the recent increase in favorable environmental conditions for algal events in central Texas, NASA DEVELOP partnered with LCRA and COA DWP to improve monitoring and early detection of algal events, utilizing satellite remote sensing and machine learning. Spatially and temporally varied chlorophyll a concentrations, cyanobacteria detections, turbidity, and water surface temperature products are used as environmental proxies. Landsat 8 Operational Land Imager (OLI) and the Sentinel-2 MultiSpectral Instrument (MSI) data were used to provide a combined revisit time of up to ~3 days and < 30 m per pixel products. Chlorophyll a concentrations were estimated using a pre-trained Mixture Density Network, and cyanobacteria detection was accomplished using the Broad Wavelength Algae Index, which can differentiate algal blooms from algal proliferations (mats). In situ data were used to validate remotely sensed measurements and quantify uncertainties. Preliminary results show a good fit between the modeled output and in situ observations, suggesting that remote sensing data can be used to retrieve biogeochemical properties and/or inherent optical properties (IOPs) of water columns in these inland human-made lakes. Uncertainties were introduced from the sensitivity to atmospheric correction, inherent mismatch between satellite and sampling data, and a relatively lower signal-to-noise ratio over water. The resulting products enable near real-time monitoring of environmental proxies relevant to algal event presence in the Highland Lakes chain, and will ultimately support water management, decision making, and risk communication.

Shuyu Chang↗
NASA NTRSDate not supplied

Training Airline Pilots for Improved Flight Path Monitoring: The Sensemaking Model Framework

The importance and benefit of improved monitoring is increasingly recognized. Improved training may be a valuable intervention. Our study (conducted 2019) assessed and trained airline First Officers on flight path monitoring skills. The exploratory study assessed monitoring pre-training in a simulator session that included monitoring challenges (8 or 7 events). A 1-hour interactive training followed, based on the Sensemaking Model of Monitoring; it presented concepts and examples using a slide deck, discussion, and simple activities. Post-training assessment used scenarios with analogous monitoring challenges (7 or 8 events) but a different setting. Performance showed significant and relatively consistent improvement. Training monitoring as sensemaking merits further investigation.

training↗