Search NASA⌕ Search

SEARCH · Search NASA

Results for “Transfer Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Cluster expansion by transfer learning for phase stability predictions

Recent progress towards universal machine-learned interatomic potentials holds considerable promise for materials discovery. Yet the accuracy of these potentials for predicting phase stability may still be limited. In contrast, cluster expansions provide accurate phase stability predictions but are computationally demanding to parameterize from first principles, especially for structures of low dimension or with a large number of components, such as interfaces or multimetal catalysts. We overcome this trade-off via transfer learning. Using Bayesian inference, we incorporate prior statistical knowledge from machine-learned and physics-based potentials, enabling us to sample the most informative configurations and to efficiently fit first-principles cluster expansions. Furthermore, this algorithm is tested on Pt:Ni, showing robust convergence of the mixing energies as a function of sample size with reduced statistical fluctuations.

36 MATERIALS SCIENCE↗

Detection of multi-reference character imbalances enables a transfer learning approach for virtual high throughput screening with coupled cluster accuracy at DFT cost

Appropriately identifying and treating molecules and materials with significant multi-reference (MR) character is crucial for achieving high data fidelity in virtual high-throughput screening (VHTS). Despite development of numerous MR diagnostics, the extent to which a single value of such a diagnostic indicates the MR effect on a chemical property prediction is not well established. We evaluate MR diagnostics for over 10 000 transition-metal complexes (TMCs) and compare to those for organic molecules. We observe that only some MR diagnostics are transferable from one chemical space to another. By studying the influence of MR character on chemical properties (i.e., MR effect) that involve multiple potential energy surfaces (i.e., adiabatic spin splitting, ΔE H–L , and ionization potential, IP), we show that differences in MR character are more important than the cumulative degree of MR character in predicting the magnitude of an MR effect. Motivated by this observation, we build transfer learning models to predict CCSD(T)-level adiabatic ΔE H–L and IP from lower levels of theory. By combining these models with uncertainty quantification and multi-level modeling, we introduce a multi-pronged strategy that accelerates data acquisition by at least a factor of three while achieving coupled cluster accuracy (i.e., to within 1 kcal mol –1 MAE) for robust VHTS.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Virtual Battery Parameter Identification using Transfer Learning based Stacked Autoencoder

Recent studies have shown that the aggregated dynamic flexibility of an ensemble of thermostatic loads can be modeled in the form of a virtual battery. The existing methods for computing the virtual battery parameters require the knowledge of the first-principle models and parameter values of the loads in the ensemble. In real-world applications, however, it is likely that the only available information are end-use measurements such as power consumption, room temperature, device on/off status, etc., while very little about the individual load models and parameters are known. We propose a transfer learning based deep network framework for calculating virtual battery state of a given ensemble of flexible thermostatic loads, from the available end-use measurements. This proposed framework extracts first order virtual battery model parameters for the given ensemble. We illustrate the effectiveness of this novel framework on different ensembles of ACs and WHs.

Stacked Autoencoder, Convolution Neural Network, v↗

Transfer learning of neural surrogates on multifidelity groundwater simulations

Multifidelity data used in the paper published in Advances in Water Resources 206 (2025) 105140, https://doi.org/10.1016/j.advwatres.2025.105140 The code used to process the data is openly available on GitHub at https://github.com/Model-Reduction-and-UQ-Group/Transfer_Learning_K_reconstruction Computationally inexpensive surrogates of process-based models, such as deep neural networks, enable ensemble-based computations used in risk assessment, data assimilation, etc. However, generation of large datasets required to train a neural network can be as expensive as the ensemble simulations themselves. We ameliorate this challenge by using data from multifidelity (MF) groundwater simulations and transfer learning (TL) to reduce data generation costs while maintaining model accuracy. As a computational example, we train a deep convolutional neural network (CNN) to reconstruct permeability fields from saturation maps derived from a multiphase flow model. Starting with very low- and low-fidelity data generated on increasingly coarse meshes, we pretrain the CNN, followed by output-layer training and fine-tuning using only a limited number of high-fidelity samples. We demonstrate the surrogate’s robustness when interpreting low-quality inputs—such as interpolated maps or data affected by noise—which has strong implications for the applicability in practical hydrogeological scenarios. This multilevel MF-TL strategy achieves a favorable trade-off between computational efficiency and predictive accuracy, significantly outperforming high-fidelity-only approaches under the same computational budget.

Chiofalo, Alessia [University of Bologna] (ORCID:0↗

Transfer Learning using Denoising Auto-Encoders for Cellular-Level Annotation of Tumor in Pathology Slides

Adversarial examples can produce altered classifications using only seemingly innocuous, imperceptible perturbations to the original image. The imperceptibility of adversarial perturbations suggests that the corresponding classifiers use decision criteria different than those of a human. In a medical setting, inexplicable decision criteria confound a pathologist’s willingness to trust machine-generated annotations. Here, we analyze denoising tumor detection models to see if they are robust to imperceptible adversarial perturbations. Moreover, to be more fully trusted by pathologists, we require tumor detectors that generate interpretable annotations which segment pathology slides into tumorous and normal regions at the cellular level. We therefore compare transfer learning based on two different autoencoder architectures, one derived from a deep denoising bottleneck autoencoder and one from an over-complete sparse autoencoder. Both autoencoders were first trained in an unsupervised manner on a set of pathology slides drawn from the Camelyon16 dataset. The latent representations produced by each autoencoder were then passed to separate neural networks that were trained in a supervised manner on binary tumor-normal masks generated by pathologists at cellular resolution. Both tumor detectors supported better than 90% AUC PR as measured by the area under the precision/recall curve on a held-out pathology slide. To assess the underlying decision criteria used by both tumor detectors, we constructed imperceptible adversarial examples which reduced the AUC PR of both models to less than 70%. Random noise of the same amplitude had almost no effect on the AUC PR of either model. Additionally, each tumor detector was resistant to adversarial “transfer” attacks targeting the other. The adversarial perturbations showed strong characteristic differences: the deep denoising models perturbations were a very diffuse, seemingly unrecognizable pattern while the sparse coding models perturbations showed traces of tissue cells.

47 OTHER INSTRUMENTATION↗

Transfer learning as a method to reproduce high-fidelity non-local thermodynamic equilibrium opacities in simulations

Simulations of high-energy density physics often need non-local thermodynamic equilibrium opacity data. These data, however, are expensive to produce at relatively low fidelity. It is even more so at high fidelity such that the opacity calculations can contribute 95 % of the total computation time. This proportion can even reach large proportions. Neural networks can be used to replace the standard calculations of low-fidelity data, and the neural networks can be trained to reproduce artificial, high-fidelity opacity spectra. In this work, it is demonstrated that a novel neural network architecture trained to reproduce high-fidelity krypton spectra through transfer learning can be used in simulations. Further, it is demonstrated that this can be done while achieving a relative per cent error of the peak radiative temperature of the hohlraum of approximately 1 % to 4 % while achieving a 19.4 $\times$ speed up.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Molecular simulation using transfer-learned potentials for the disordered nanoscale structure of nitrogen-doped nanoporous carbons

Machine learning (ML)-based molecular dynamics (MD) simulations of the formation of a class of N-doped nanoporous carbons are performed to assess their disordered partially graphitized nanoscale structure. The study is motivated by the effectiveness of so-called nitrogen assembly carbons (NACs) for catalysis applications. Benchmark simulations for pure-C disordered graphitic systems reveal the importance of reliably capturing the vdW component of the potentials in order to accurately describe the tendency for layering of disordered graphene-like sheets. In our modeling, this is achieved by a transfer learning strategy incorporating features of the energetics from the optB88-vdW DFT functional into potentials initially trained with a less expensive functional, thereby providing a superior description of the pure-C systems. Generation from MD simulations of realistic partially graphitized structures is significantly more challenging for N-doped versus for pure C systems. However, such structures are achieved by a tailored MD simulation protocol mimicking the experimental synthesis process and in particular incorporating an annealing and subsequent quenching stages. Simulated PXRD patterns effectively reproduce the features of experimental observations for NACs, including the appearance of a prominent but broad (002) peak at around 25, and the development of another weaker feature associated with in-layer ordering of mixed C-N graphene-like sheets.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Geolocation tracking for human identification and activity recognition using radar deep transfer learning

Abstract Human identification and activity recognition (HIAR) is crucial for many applications, such as surveillance, smart homes, and assisted living. As a sensing modality, radar has many unique characteristics including privacy protection, and contactless sensing. Single classification systems have shown to be accurate, but for long‐term solutions both human identification (ID) and human activity recognition (HAR) will need to be integrated in one system where it can be utilised simultaneously. In this article, a novel radar‐based human tracking system is presented where three classifiers are utilised to identify the subject and his/her behaviour. For any kind of motion, the system tracks the subject and detect the type of his/her motion. Based on the detected type of motion, the three classifiers are utilised for identification and activity recognition. The classifiers are built utilising deep transfer learning where three radar datasets are established to train and validate each of the deep networks. To recognise six activities and 10 human subjects, the three classifiers, namely, HAR, Gait ID, and Heart sound ID, achieve superior performance compared to the best reported results in literature with classification accuracies of 97.6%, 100%, and 41.8% respectively. Three successful examples are presented to demonstrate the introduced concept.

Alkasimi, Ahmad↗

Cross-Feature Transfer Learning for Efficient Tensor Program Generation

Tuning tensor program generation involves navigating a vast search space to find optimal program transformations and measurements for a program on the target hardware. The complexity of this process is further amplified by the exponential combinations of transformations, especially in heterogeneous environments. This research addresses these challenges by introducing a novel approach that learns the joint neural network and hardware features space, facilitating knowledge transfer to new, unseen target hardware. A comprehensive analysis is conducted on the existing state-of-the-art dataset, TenSet, including a thorough examination of test split strategies and the proposal of methodologies for dataset pruning. Leveraging an attention-inspired technique, we tailor the tuning of tensor programs to embed both neural network and hardware-specific features. Notably, our approach substantially reduces the dataset size by up to 53% compared to the baseline without compromising Pairwise Comparison Accuracy (PCA). Furthermore, our proposed methodology demonstrates competitive or improved mean inference times with only 25–40% of the baseline tuning time across various networks and target hardware. The attention-based tuner can effectively utilize schedules learned from previous hardware program measurements to optimize tensor program tuning on previously unseen hardware, achieving a top-5 accuracy exceeding 90%. This research introduces a significant advancement in autotuning tensor program generation, addressing the complexities associated with heterogeneous environments and showcasing promising results regarding efficiency and accuracy.

97 MATHEMATICS AND COMPUTING↗

Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning

Computational modeling of chemical and biological systems at atomic resolution is a crucial tool in the chemist’s toolset. The use of computer simulations requires a balance between cost and accuracy: quantum-mechanical methods provide high accuracy but are computationally expensive and scale poorly to large systems, while classical force fields are cheap and scalable, but lack transferability to new systems. Machine learning can be used to achieve the best of both approaches. Here we train a general-purpose neural network potential (ANI-1ccx) that approaches CCSD(T)/CBS accuracy on benchmarks for reaction thermochemistry, isomerization, and drug-like molecular torsions. This is achieved by training a network to DFT data then using transfer learning techniques to retrain on a dataset of gold standard QM calculations (CCSD(T)/CBS) that optimally spans chemical space. The resulting potential is broadly applicable to materials science, biology, and chemistry, and billions of times faster than CCSD(T)/CBS calculations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Improving radiograph analysis throughput through transfer learning and object detection

SIGN Fracture Care International partners with surgeons in low-resource hospitals worldwide to provide access to effective orthopedic care by donating educational materials and innovatively designed surgical implants. Over two decades, SIGN’s Online Surgical Database (SOSD) has grown to contain over 500,000 medical images, with radiographs holding the majority share. One challenge in working with hospitals worldwide is that both the radiographs uploaded to the SOSD and the data entry accompanying the uploads vary in quality. To improve the accuracy of data in the SOSD, we trained a model to detect surgical implants in radiographs. We first developed a tool to automatically detect radiographs, then trained an object detection model to determine the number and placement of surgical implants visible in the radiograph. Active learning was used to generate a training set containing 2,510 radiographs with screws, nails, and plates labeled by bounding boxes. Training a model to simultaneously recognize all three classes of implants gave a low average precision (AP) for the plate class, likely due to the low number of plate instances in our training set and the large variety of surgical plates used by SIGN-partnered surgeons. Applying standard image augmentation techniques to increase the plate count in our training set did not appreciably increase the AP of plate detection. To improve plate detection, we redrew the bounding boxes to account for correlations between the screw and plate classes. Training one model to detect nails and screws and a separate model to detect plates increased the AP of plate detection by 78.8 percentage points. The AP of each class was 80.7% for screws, 93.6% for nails, and 92.6% for plates; meanwhile, the sensitivity was 92% for screws, 86% for nails, and 81% for plates. We show that object detection methods can be used to detect surgical implants in radiographs of varying quality; however, the detection ability is dependent on the type of implant, and some implants, in our case plates, must be treated differently than others. Such tools can improve the throughput of radiograph analysis, assisting physicians and surgeons with the treatment of bone fractures.

60 APPLIED LIFE SCIENCES↗

Active and Transfer Learning of High-Dimensional Neural Network Potentials for Transition Metals

Classical molecular dynamics (MD) simulations represent a very popular and powerful tool for materials modeling and design. The predictive power of MD hinges on the ability of the interatomic potential to capture the underlying physics and chemistry. There have been decades of seminal work on developing interatomic potentials, albeit with a focus predominantly on capturing the properties of bulk materials. Such physics-based models, while extensively deployed for predicting the dynamics and properties of nanoscale systems over the past two decades, tend to perform poorly in predicting nanoscale potential energy surfaces (PESs) when compared to high-fidelity first-principles calculations. These limitations stem from the lack of flexibility in such models, which rely on a predefined functional form. Machine learning (ML) models and approaches have emerged as a viable alternative to capture the diverse size-dependent cluster geometries, nanoscale dynamics, and the complex nanoscale PESs, without sacrificing the bulk properties. Here, in this study, we introduce an ML workflow that combines transfer and active learning strategies to develop high-dimensional neural networks (NNs) for capturing the cluster and bulk properties for several different transition metals with applications in catalysis, microelectronics, and energy storage, to name a few. Our NN first learns the bulk PES from the high-quality physics-based models in literature and subsequently augments this learning via retraining with a higher-fidelity first-principles training data set to concurrently capture both the nanoscale and bulk PES. Our workflow departs from status-quo in its ability to learn from a sparsely sampled data set that nonetheless covers a diverse range of cluster configurations from near-equilibrium to highly nonequilibrium as well as learning strategies that iteratively improve the fingerprinting depending on model fidelity. All the developed models are rigorously tested against an extensive first-principles data set of energies and forces of cluster configurations as well as several properties of bulk configurations for 10 different transition metals. Our approach is material agnostic and provides a methodology to transfer and build upon the learnings from decades of seminal work in molecular simulations on to a new generation of ML-trained potentials to accelerate materials discovery and design.

36 MATERIALS SCIENCE↗

A universal density matrix functional from molecular orbital-based machine learning: Transferability across organic molecules

We address the degree to which machine learning (ML) can be used to accurately and transferably predict post-Hartree-Fock correlation energies. Refined strategies for feature design and selection are presented, and the molecular-orbital-based machine learning (MOB-ML) method is applied to several test systems. Strikingly, for the second-order Møller-Plessett perturbation theory, coupled cluster with singles and doubles (CCSD), and CCSD with perturbative triples levels of theory, it is shown that the thermally accessible (350 K) potential energy surface for a single water molecule can be described to within 1 mhartree using a model that is trained from only a single reference calculation at a randomized geometry. To explore the breadth of chemical diversity that can be described, MOB-ML is also applied to a new dataset of thermalized (350 K) geometries of 7211 organic models with up to seven heavy atoms. In comparison with the previously reported Δ-ML method, MOB-ML is shown to reach chemical accuracy with threefold fewer training geometries. Finally, a transferability test in which models trained for seven-heavy-atom systems are used to predict energies for thirteen-heavy-atom systems reveals that MOB-ML reaches chemical accuracy with 36-fold fewer training calculations than Δ-ML (140 vs 5000 training calculations).

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Transfer Learning for HVAC System Fault Detection

Faults in HVAC systems degrade thermal comfort and energy efficiency in buildings and have received significant attention from the research community, with data driven methods gaining in popularity. Yet the lack of labeled data, such as normal versus faulty operational status, has slowed the application of machine learning to HVAC systems. In addition, for any particular building, there may be an insufficient number of observed faults over a reasonable amount of time for training. To overcome these challenges, we present a transfer methodology for a novel Bayesian classifier designed to distinguish between normal operations and faulty operations. The key is to train this classifier on a building with a large amount of sensor and fault data (for example, via simulation or standard test data) then transfer the classifier to a new building using a small amount of normal operations data from the new building. We demonstrate a proof-of-concept for transferring a classifier between architecturally similar buildings in different climates and show few samples are required to maintain classification precision and recall.

transfer learning, Building HVAC, Bayesian framewo↗

Enhancing Cluster Identification in Atom Probe Tomography Data Using Transfer Learning

Atom probe tomography (APT) has enabled the direct visualization of solute clusters, providing valuable insights into material structures. This clustering is crucial for understanding the nanoscale composition and behavior of materials, which can significantly influence their mechanical and physical properties. However, the widely used clustering methods in the APT community face challenges such as subjective parametric selection and limited applicability, particularly in dealing with overlapping clusters, nested clusters, and artifacts across different scales, such as precipitates and dislocations. To address these challenges, we present a framework based on density-based cluster analysis that aims to be less dependent on user input, reproducible, and robust.

Density-based clustering↗