Search NASA⌕ Search

SEARCH · Search NASA

Results for “Transfer Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Online transfer learning strategy for enhancing the scalability and deployment of deep reinforcement learning control in smart buildings

In recent years, advanced control strategies based on Deep Reinforcement Learning (DRL) proved to be effective in optimizing the management of integrated energy systems in buildings, reducing energy costs and improving indoor comfort conditions when compared to traditional reactive controllers. However, the scalability and implementation of DRL controllers are still limited since they require a considerable amount of time before converging to a near-optimal solution. This issue is currently addressed in literature through the offline pre-training of the DRL agent. However this solution results in two main critical issues: (1) the need to develop a building surrogate model to perform the training task, and (2) the need to perform a fine-tuning process over several training episodes to obtain a near-optimal control policy. In this context, this paper introduces an Online Transfer Learning (OTL) strategy that exploits two knowledge-sharing techniques, weight-initialization and imitation learning, to transfer a DRL control policy from a source office building to various target buildings in a simulation environment coupling EnergyPlus and Python. A DRL controller based on discrete Soft Actor–Critic (SAC) is trained on the source building to manage the operation of a cooling system consisting of a chiller and a thermal storage. Several target buildings are defined to benchmark the performance of the OTL strategy with that of a Rule-Based Controller (RBC) and two DRL-based control strategies, deployed in offline and online fashion. The strategy adopted for OTL emulates the real world implementation with a simulation process by implementing the transferred DRL agent for a single episode in the target buildings. Target buildings have the same geometrical features and are served by the same energy system as the source building, but differ in terms of weather conditions, electricity price schedules, occupancy patterns, and building envelope efficiency levels. The results show that the OTL strategy can reduce the cumulated sum of temperature violations on average by 50% and 80% respectively when compared to RBC and online DRL while enhancing the energy system operation with electricity cost savings ranging between 20% and 40%. Furthermore, the OTL agent performs slightly worse than the offline DRL controller but it does not require any modeling effort and can be implemented directly on target buildings emulating a real-world implementation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Sharing is caring: An extensive analysis of parameter-based transfer learning for the prediction of building thermal dynamics

In recent years deep neural networks have been proposed as a lightweight data-driven model to capture high-dimensional, nonlinear physical processes to predict building thermal responses. However, the need of a large amount of data for the training process of deep neural networks clashes with the potential limited data availability in most existing or new buildings. Transfer learning aims to enhance the performance of a target learner exploiting knowledge from related and similar environments. This study conducted a suite of experiments that leveraged 250 data-driven models based on a synthetic dataset of a building archetype to study the influence of data availability, energy efficiency level, occupancy and climate for the transfer process of thermal dynamics. The performance of the transfer learning process was compared against a classical machine learning approach. Here, the results suggest that building thermal dynamics can be effectively transferred under the same climatic conditions, increasing performance when dealing with different occupancy schedules, efficiency levels and low data availability. Furthermore, the paper compares the performance of both transfer learning and machine learning approaches in an online fashion, to support the implementation in real-world deployment.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Black Swan Event Small-Sample Transfer Learning (BEST-L) and its Case Study on Electrical Power Prediction in COVID-19

The black swan event will usually cause a great impact on the normal operation of society. The scarcity of such events leads to a lack of relevant data and challenges in dealing with related problems. Different situations also make the traditional methods invalid. In this paper, a transfer learning framework and a convolutional neuron network are proposed to deal with the black swan small-sample events (BEST-L). Taking the COVID-19 as a typical black swan event, the BEST-L is utilized to achieve accurate mid-term load forecasting using the relationship between economy and electricity consumption. The experiment results show that the transfer learning model can effectively learn the basic knowledge about the relationship between the adopted input and output data and use a relatively small amount of data during the black swan event to improve the target areas' generalization. The approach and results can provide an effective approach to respond and react to sudden changes quickly and effectively in similar open problems.

black swan event↗
DOE OSTI2019

Falls Risk Classification of Older Adults Using Deep Neural Networks and Transfer Learning

Prior research in falls risk classification using inertial sensors has relied on the use of engineered features, which has concluded in a feature space containing hundreds of features that are likely redundant and possibly irrelevant. In this paper, we propose using fully convolutional neural networks (FCNNs) to classify older adults at low or high risk of falling using inertial sensor data collected from a smartphone. Due to the limited nature of older adult inertial gait data sets, we first pre-train the FCNN models using a publicly available data set for pedestrian activity recognition. Then via transfer learning, we train the network for falls risk classification. We reveal that via transfer learning, our falls risk classifier obtains an area under the receiver operating characteristic curve of 93.3%, which is 10.6% higher than the equivalent model trained without the use of transfer learning. Moreover, we show that our method outperforms other standard machine learning classifiers trained on features developed in prior research.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

Active- and transfer-learning applied to microscale-macroscale coupling to simulate viscoelastic flows

Active- and transfer-learning are applied to microscale dynamics of polymer flows for the multiscale discovery of effective constitutive approximations required in viscoelastic flow simulation. The result is macroscopic rheology directly connected to a microstructural model. Micro and macroscale simulations are adaptively coupled by means of Gaussian process regression (GPR) to run the expensive microscale computations only as necessary. This multiscale method is demonstrated with flows of a polymer solution as a model system. At the microscale level dissipative particle dynamics (DPD) is employed to model the fluid as a suspension of bead-spring micro-structures subjected to steady shear flow. The results yield the non-Newtonian viscosity and the first normal stress difference at strain rates as training data used in a GPR model. DPD parameters are calibrated with respect to experimental data for a real polymer solution. Compliance with these data requires adjustment of the DPD model's cutoff radius, which then becomes a function of the second invariant of the strain rate tensor. The FENE-P model is chosen for the macroscale description using the spectral element method (SEM) to simulate channel flow and flow past a circular cylinder. The DPD results at the lowest possible shear strain rate yield an estimate of the zero-shear rate viscosity, which allows the initiation of the macroscale flow by SEM as a Newtonian fluid. The resulting strain-rate field is surveyed to determine additional shear strain rate sampling points for the DPD system. This new information allows an initial fitting of parameters of the constitutive equation followed by new SEM simulations at the macroscale. Additionally, guided by active-learning GPR to select new sampling points, this process continues until convergence is achieved. The effectiveness of this new simulation paradigm for viscoelastic flows is tested with different macroscale operating conditions. The effective closure learned in the channel simulation is then transferred directly to the flow past a circular cylinder at low Reynolds number, where the results show that only two additional DPD simulations are required to achieve a satisfactory constitutive model. With an increase of the Reynolds number, the active-learning scheme automatically detects the inaccuracy of the learned constitutive model, and initiates additional DPD simulations for the extra data needed to once again close the microscale-macroscale coupled system. This new paradigm of active- and transfer-learning for multiscale modeling is readily applicable to other microscale-macroscale coupled simulations of complex fluids and other materials. Furthermore, the coupling between microscale and macroscale solvers can be seamlessly implemented with our open source multiscale universal interface (MUI) library.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Driver Distraction Behavior Detection using a Vision Transformer Model based on Transfer Learning Strategy

Driver distraction behavior is one of the critical factors in traffic accidents. Thus, advanced driver state detection system has become the focus in the field of intelligent vehicle. However, in practical applications, insufficient samples of driving distraction behaviors bring great challenges to training a personalized behavior distraction detection model for a specific driver. To this end, a novel transformer model based on a transfer learning strategy is proposed in this paper to accurately recognize driver distraction behavior. Inspired by the effect of the transformer network in visual recognition, we firstly present a transformer behavior distraction detection system to identify the behavior categories that cause driver distraction. Then, for the specific driving dataset in practical application scenarios, the transfer learning strategy is introduced into the driver distraction detection model to further train the general transformer network. The effectiveness of the transformer based on the transfer learning strategy is validated compared with other traditional deep learning methods. The results show that the proposed detection method has better generalization ability and higher accuracy.

Fang, Zhenwu↗
DOE OSTI2024

Data for The utility of transfer learning to improve the performance of deep learning in axon segmentation

The utility of transfer learning to improve the performance of deep learning in axon segmentation Data Data: All the input and labeled volumes tf-logs: Tensorflow logs, view with command "tensorboard --logdir [name of folder]" Model Weights: model_weights: the argument list under variable combo indicate 1) no oversampling, 2) no rotation, 3) no learn scheduler, and 4) flipping on all three dimensions, and the additional values indicate 5) elastic deformation percentage, 6) rotate deformation percentage, 7) layer setting , 8) learning rate, and 9) training/validation/test data division suffix (leave '' if not using suffix). Results: Output from inference segment_total_results_validation_final: All validation results and calculations segment_total_results: All test results and calculations Authors The modified code was created for a paper by: Marjolein Oostrom, Michael A. Muniak, Rogene Eichler West, Sarah Akers, Paritosh Pande, Moses Obiri, Wei Wang, Kasey Bowyer, Zhuhao Wu, Lisa Bramer, Tianyi Mao, Bobbie Jo Webb-Robertson The work is adapted from Github TrailMap, which was created by Albert Pun and Drew Friedmann Acknowledgments MO, RMEW, SA, MO, LB, BJWR were supported by the Laboratory Directed Research and Development at Pacific Northwest National Laboratory (PNNL), a Department of Energy facility operated by Battelle under contract DE-AC05-76RLO01830. WW, KB, and ZW were supported in part by a NIH/BRAIN Initiative Grant RF1MH128969. MAM and TM were supported by two NIH/BRAIN Initiative Grants R01NS104944, RF1MH120119 and NIH R01NS081071. This research is affiliated with the Pacific northwest bioMedical Innovation Co-laboratory (PMedIC) collaboration between OHSU and PNNL.

Oostrom, Marjolein T↗
DOE OSTI2019

Deep Transfer Learning With Field-Based Measurements for Large Area Classification

Classification of Electro-Optical (EO) datasets using Deep Neural Networks (DNNs) has lead to high-performance supervised learning algorithms. However, DNNs require a large amount of labeled data for training which often have limited availability in ecological studies. Up-to-date classification of vegetation in sensitive Arctic ecosystems continue to be a challenge. In an ecosystem undergoing rapid change, capturing the dynamics of vegetation requires the existing maps of vegetation (such as Alaska Existing Vegetation Type (AKEVT), circa 2000) to be updated based on frequent field based observation of vegetation. A method is needed to transfer the knowledge gained from field-collected observations to a larger area using DNNs. Transfer learning is a machine learning technique where a model trained on one task is re-purposed on another similar task. This paper seeks to train DNNs using field-collected observations and apply transfer learning to apply the knowledge gained to update existing vegetation map at larger scale. We test two DNN methods, (1) a deep multilayer perceptron (MLP) model and (2) siamese MLP network that uses a structure to rank similarity between inputs and can be used for training datasets with few samples and show good performance with limited datasets (e.g. few-shot learning). The results show ∼90% accuracy (using the field observations for evaluation) when transfer learning is applied to a siamese network, compared to ∼45% accurate when a MLP is trained on the AKEVT and evaluated on the field observations. The approach show promise for improving and update the existing vegetation maps over large areas using limited field-based observations.

Langford, Zach↗
DOE OSTI2022

Short-term solar radiation forecast using total sky imager via transfer learning

Ground-based sky cameras, which capture hemispherical images, have been extensively used for localized monitoring of clouds. This paper proposes a short-term forecasting approach based on transfer learning using Total Sky-Imager (TSI) images of the Southern Great Plains (SGP) site obtained from the Atmospheric Radiation Measurement (ARM) dataset. An accurate estimation of solar irradiance using TSI is key for short-term solar energy generation forecasting and optimal energy consumption planning. We make use of deep neural network architectures such as AlexNet and ResNet-101 to extract the underlying deep convolution features from TSI images and then train using an ensemble learning approach to model and forecast solar radiation. We demonstrate the performance of the proposed approach by showcasing the best and worst cases. Thus, the transfer learning approach significantly reduces the time and resources required for modeling solar radiation. We outperform with reference to another state-of-art technique for solar modeling using TSI images at different forecast lead times.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Using deep transfer learning and satellite imagery to estimate urban air quality in data-poor regions

Urban air pollution is a critical public health challenge in low-and-middle-income countries (LMICs). At the same time, LMICs tend to be data-poor, lacking adequate infrastructure to monitor air quality (AQ). As LMICs undergo rapid urbanization, the socio-economic burden of poor AQ will be immense. Here we present a globally scalable two-step deep learning (DL) based approach for AQ estimation in LMIC cities that mitigates the need for extensive AQ infrastructure on the ground. We train a DL model that can map satellite imagery to AQ in high-income countries (HICs) with sufficient ground data, and then adapt the model to learn meaningful AQ estimates in LMIC cities using transfer learning. The trained model can explain up to 54% of the variation in the AQ distribution of the target LMIC city without the need for target labels. The approach is demonstrated for Accra in Ghana, Africa, with AQ patterns learned and adapted from two HIC cities, specifically Los Angeles and New York.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Multisource Mobile Transfer Learning Algorithm Based on Dynamic Model Compression

With the development of the Internet of Things, the application of computer vision on mobile phones is becoming more and more extensive and people have higher and higher requirements for the timeliness of the recognition results returned and the processing capabilities of the mobile phone for image recognition. However, the processing capability and storage capability of the user terminal equipment cannot meet the needs of identifying and storing a large number of pictures, and the data transmission process will cause high energy consumption of the terminal equipment. At the same time, multisource deep transfer learning has outstanding performance in computer vision and image classification. However, due to the huge amount of calculation of the deep network model, it is impossible to use the existing excellent network model to realize image recognition and classification on the mobile terminal. In order to solve the abovementioned problems, we propose a multisource mobile transfer learning algorithm based on dynamic model compression, this algorithm considers the realization of multisource transfer learning computing in the case of multiple mobile device computing source domains, and the method also guarantees data privacy and security for each device (origin domain). Meanwhile, extensive experiments show that our method can achieve remarkable results in popular image classification datasets.

Gao, Peng↗
DOE OSTI2021

Predicting fault slip via transfer learning

Abstract Data-driven machine-learning for predicting instantaneous and future fault-slip in laboratory experiments has recently progressed markedly, primarily due to large training data sets. In Earth however, earthquake interevent times range from 10’s-100’s of years and geophysical data typically exist for only a portion of an earthquake cycle. Sparse data presents a serious challenge to training machine learning models for predicting fault slip in Earth. Here we describe a transfer learning approach using numerical simulations to train a convolutional encoder-decoder that predicts fault-slip behavior in laboratory experiments. The model learns a mapping between acoustic emission and fault friction histories from numerical simulations, and generalizes to produce accurate predictions of laboratory fault friction. Notably, the predictions improve by further training the model latent space using only a portion of data from a single laboratory earthquake-cycle. The transfer learning results elucidate the potential of using models trained on numerical simulations and fine-tuned with small geophysical data sets for potential applications to faults in Earth.

58 GEOSCIENCES↗
DOE OSTI2024

A data-driven framework for predicting machining stability: employing simulated data, operational modal analysis, and enhanced transfer learning

Chatter, a self-excited vibration phenomenon, presents a significant challenge in machining operations, particularly in high-speed milling, where it can degrade tool life, reduce material removal efficiency, and compromise workpiece quality. Addressing this challenge requires a reliable predictive model that can accommodate the complex dynamics of various machining scenarios. This study introduces a novel, data-driven approach to predicting machining stability, leveraging over 140,000 simulated datasets and employing advanced techniques such as operational modal analysis (OMA), enhanced transfer learning (TL), and receptance coupling substructure analysis (RCSA). By integrating these methodologies, the framework effectively classifies and predicts chatter across diverse operational modes, achieving robust and accurate outcomes. Our model utilizes a Random Forest (RF) classifier trained with the comprehensive dataset, which demonstrates substantial improvements in both predictive accuracy and robustness. Specifically, the RF model achieved an accuracy rate of 85%, an area under the curve (AUC) of 0.90, and an F1 score of 0.88, underscoring its capability to adapt to varying machining configurations. These results highlight the framework’s potential to enhance operational efficiency and machining quality by providing reliable chatter predictions across a broad range of machining parameters. In conclusion, this research thus offers a significant advancement in predictive maintenance for machining processes, enabling more stable and efficient manufacturing operations.

42 ENGINEERING↗
DOE OSTI2025

Transfer learning nonlinear plasma dynamic transitions in low dimensional embeddings via deep neural networks

Deep learning algorithms provide a new paradigm to study high-dimensional dynamical behaviors, such as those in fusion plasma systems. Development of novel, data-driven model reduction methods, coupled with detection of abnormal modes with plasma physics, opens a unique opportunity to identify plasma instabilities through automated construction of parsimonious models that can be tuned to balance accuracy and cost. Our fusion transfer learning (FTL) model demonstrates success in rapidly reconstructing nonlinear kink mode structures by learning from a limited amount of nonlinear simulation data. The knowledge transfer process leverages a pre-trained neural encoder–decoder network, initially trained on linear simulations, to effectively capture nonlinear dynamics. The low-dimensional embeddings extract the coherent structures of interest, while preserving the inherent dynamics of the complex system. Experimental results highlight FTL’s capacity to capture transitional behaviors and dynamical features in plasma dynamics—a task often challenging for conventional methods. The model developed in this study is generalizable and can be extended broadly through transfer learning to address various magnetohydrodynamics modes.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Power System Event Identification with Transfer Learning Using Large-scale Real-world Synchrophasor Data in the United States

The lack of sufficient labeled events and long training time limit the applicability of deep neural network-based power system event identification using synchrophasor data. In this paper, we propose to leverage transfer learning technique to boost the reliability and reduce the required training time of neural classifier for power system event identification. We use the weights of a neural classifier trained on one transmission system as the initial parameters of another neural classifier for a different transmission system. Numerical tests with real-world synchrophasor data from the Eastern and Western Interconnections of the United States show that the proposed transfer learning approach is very effective in not only improving the training reliability but also reducing the training time.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2018

Sparse coding of pathology slides compared to transfer learning with deep neural networks

Background Histopathology images of tumor biopsies present unique challenges for applying machine learning to the diagnosis and treatment of cancer. The pathology slides are high resolution, often exceeding 1GB, have non-uniform dimensions, and often contain multiple tissue slices of varying sizes surrounded by large empty regions. The locations of abnormal or cancerous cells, which may constitute a small portion of any given tissue sample, are not annotated. Cancer image datasets are also extremely imbalanced, with most slides being associated with relatively common cancers. Since deep representations trained on natural photographs are unlikely to be optimal for classifying pathology slide images, which have different spectral ranges and spatial structure, we here describe an approach for learning features and inferring representations of cancer pathology slides based on sparse coding. Results We show that conventional transfer learning using a state-of-the-art deep learning architecture pre-trained on ImageNet (RESNET) and fine tuned for a binary tumor/no-tumor classification task achieved between 85% and 86% accuracy. However, when all layers up to the last convolutional layer in RESNET are replaced with a single feature map inferred via a sparse coding using a dictionary optimized for sparse reconstruction of unlabeled pathology slides, classification performance improves to over 93%, corresponding to a 54% error reduction. Conclusions We conclude that a feature dictionary optimized for biomedical imagery may in general support better classification performance than does conventional transfer learning using a dictionary pre-trained on natural images.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Leveraging transfer learning and leaf spectroscopy for leaf trait prediction with broad spatial, species, and temporal applicability

Accurate and reliable prediction of leaf traits is crucial for understanding plant adaptations to environmental variation, monitoring terrestrial ecosystems, and enhancing comprehension of functional diversity and ecosystem functioning. Currently, various approaches (e.g., statistical, physical models) have been developed to estimate leaf traits through hyperspectral remote sensing and leaf spectroscopy. However, the absence of high-performing, transferable, and stable models across various domains of space, plant functional types (PFTs) and seasons hinder our ability to quantify and comprehend spatiotemporal variations in leaf traits. This study proposes robust and highly transferable models for better predicting leaf traits with hyperspectral reflectance. Initially, three datasets were assembled, pairing common leaf traits — chlorophyll (Chla+b), carotenoids (Ccar), leaf mass per area (LAM), equivalent water thickness (EWT) — with leaf spectra measurements collected across diverse geographic locations in the U.S. and Europe, PFTs, and seasons. Measurements were acquired using spectroradiometers (e.g., ASD FieldSpec 3/4/Pro and SVC HR-1024i) with integrating spheres, leaf clips, and contact probes. Here, we then developed transfer learning-based hybrid models that incorporated the domain knowledge of radiative transfer models (RTMs) through pretraining processes and were well-constrained by fine-tuning with field measurements. Through comparison with other state-of-the-art statistical models, including partial-least squares regression (PLSR) and Gaussian Process Regression (GPR), as well as pure physical models, we found that the proposed transfer learning models achieved better predictive performance and higher transferability. Specifically, compared to other statistical models and pure RTMs, the transfer learning model exhibited higher coefficient of determination (R 2 ) values with range of 0.01 to 0.79, lower normalized root mean square error (NRMSE) with range of 0.06 % to 33.25 % in model performance. Additionally, the models exhibited improved transferability, with higher R 2 values range from 0.04 to 0.32, lower NRMSE range from 0.08 % to 30.81 %. The findings underscore that transfer learning models through integrating domain knowledge from RTMs and limited observations, can harness the advantages of both RTMs and statistical models and serve as a promising approach for effectively predicting leaf traits.

59 BASIC BIOLOGICAL SCIENCES↗