SEARCH · Search NASA

Results for “fine tuning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Precise Fermi level engineering in a topological Weyl semimetal via fast ion implantation

The precise controllability of the Fermi level is a critical aspect of quantum materials. For topological Weyl semimetals, there is a pressing need to fine-tune the Fermi level to the Weyl nodes and unlock exotic electronic and optoelectronic effects associated with the divergent Berry curvature. However, in contrast to two-dimensional materials, where the Fermi level can be controlled through various techniques, the situation for bulk crystals beyond laborious chemical doping poses significant challenges. Here, we report the milli-electron-volt (meV) level ultra-fine-tuning of the Fermi level of bulk topological Weyl semimetal tantalum phosphide using accelerator-based high-energy hydrogen implantation and theory-driven planning. By calculating the desired carrier density and controlling the accelerator profiles, the Fermi level can be experimentally fine-tuned from 5 meV below, to 3.8 meV below, to 3.2 meV above the Weyl nodes. High-resolution transmission electron microscopy reveals the crystalline structure is largely maintained under irradiation, while electrical transport indicates that Weyl nodes are preserved and carrier mobility is also largely retained. Our work demonstrates the viability of this generic approach to tune the Fermi level in semimetal systems and could serve to achieve property fine-tuning for other bulk quantum materials with ultrahigh precision.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Load Profile Inpainting for Missing Load Data Restoration and Baseline Estimation

This paper introduces a Generative Adversarial Nets (GAN) based, Load Profile Inpainting Network (Load-PIN) for restoring missing load data segments and estimating the baseline for a demand response event. The inputs are time series load data before and after the inpainting period together with explanatory variables (e.g., weather data). Here, we propose a Generator structure consisting of a coarse network and a fine-tuning network. The coarse network provides an initial estimation of the data segment in the inpainting period. The fine-tuning network consists of self-attention blocks and gated convolution layers for adjusting the initial estimations. Loss functions are specially designed for the fine-tuning and the discriminator networks to enhance both the point-to-point accuracy and realisticness of the results. We test the Load-PIN on three real-world data sets for two applications: patching missing data and deriving baselines of conservation voltage reduction (CVR) events. We benchmark the performance of Load-PIN with five existing deep-learning methods. Our simulation results show that, compared with the state-of-the-art methods, Load-PIN can handle varying-length missing data events and achieve 15-30% accuracy improvement.

14 SOLAR ENERGY↗

DOE OSTI2022

Conformational spread drives the evolution of the calcium–calmodulin protein kinase II

The calcium calmodulin (Ca 2+ /CaM) dependent protein kinase II (CaMKII) decodes Ca 2+ frequency oscillations. The CaMKIIα isoform is predominantly expressed in the brain and has a central role in learning. I matched residue and organismal evolution with collective motions deduced from the atomic structure of the human CaMKIIα holoenzyme to learn how its ring architecture abets function. Protein dynamic simulations showed its peripheral kinase domains (KDs) are conformationally coupled via lateral spread along the central hub. The underlying β-sheet motions in the hub or association domain (AD) were deconvolved into dynamic couplings based on mutual information. They mapped onto a coevolved residue network to partition the AD into two distinct sectors. A second, energetically stressed sector was added to ancient bacterial enzyme dimers for assembly of the ringed hub. The continued evolution of the holoenzyme after AD–KD fusion targeted the sector’s ring contacts coupled to the KD. Among isoforms, the α isoform emerged last and, it alone, mutated rapidly after the poikilotherm–homeotherm jump to match the evolution of memory. The correlation between dynamics and evolution of the CaMKII AD argues single residue substitutions fine-tune hub conformational spread. The fine-tuning could increase CaMKIIα Ca 2+ frequency response range for complex learning functions.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2026

PRIME: An evaluation framework for protein representation inference and generalization in viral mutation space

Background Protein language models (PLMs) have revolutionized protein fitness prediction, yet their application to rapidly evolving viral pathogens is often confounded by extreme sequence homology. This homology leads to “data leakage” in standard random validation splits, yielding inflated performance metrics that fail to translate into real-world biosurveillance utility. Results We present Protein Representation Inference for Mutation Evaluation (PRIME), a framework that integrates domain-specific fine-tuning with a rigorous position-stratified validation protocol to evaluate viral threats. Using a dataset of 347,432 SARS-CoV-2 receptor binding domain (RBD) sequences, we demonstrate that while random training data split yields deceptive R 2 values (> 0.90), they fail to generalize to novel mutational sites. By benchmarking models up to 650 M parameters, we show that domain-specific fine-tuning of the ESM-C 600 M model with correctly stratified data provides an initial demonstration of predictive signal for binding affinity and expression at unseen mutational sites of binding affinity and expression on unseen sites (R 2 ~0.23), a significant advancement over base foundation models which exhibit no predictive power (R 2 <0). PRIME’s embedding-based clustering identified 3.03% of bat coronavirus sequences as candidates for further experimental prioritization based on their functional similarity to human-infective strains in embedding space, offering a perspective complementary to traditional phylogenetic methods. Conclusion PRIME establishes a new benchmark for the application of PLMs in pathogen surveillance. Our findings demonstrate that state-of-the-art models and fine-tuning, when paired with stratified validation, provide biologically meaningful insights into pathogen evolution and zoonotic risk.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

What can solve the strong CP problem?

Three possible strategies have been advocated to solve the strong CP problem. The first is the axion, a dynamical mechanism that relaxes any initial value of the CP violating angle $\overline{θ}$ to zero. The second is the imposition of new symmetries that are believed to set $\overline{θ}$ to zero in the UV. The third is the acceptance of the fine tuning of parameters. We argue that the latter two solutions do not solve the strong CP problem. The θ term of QCD is not a parameter — it does not exist in the Hamiltonian. Rather, it is a property of the quantum state that our universe finds itself in, arising from the fact that there are CP violating states of a CP preserving Hamiltonian. It is not eliminated by imposing parity as a symmetry since the underlying theory is already parity symmetric and that does not preclude the existence of CP violating states. Moreover, since the value of θ realized in our universe is a consequence of measurement, it is inherently random and cannot be fine tuned by choice of parameters. Rather any fine tuning would require a tuning between parameters in the theory and the random outcome of measurement. Our results considerably strengthen the case for the existence of the axion and axion dark matter. The confusion around θ arises from the fact that unlike classical mechanics, the Hamiltonian and Lagrangian are not equivalent in quantum mechanics. The Hamiltonian defines the differential time evolution, whereas the Lagrangian is a solution to this evolution. Consequently, initial conditions could in principle appear in the Lagrangian but not in the Hamiltonian. This results in aspects of the initial condition such as θ misleadingly appearing in the Lagrangian as parameters. We comment on the similarity between the θ vacua and the violations of the constraint equations of classical gauge theories in quantum mechanics.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2021

Identity Document to Selfie Face Matching Across Adolescence

Matching live images (“selfies”) to images from ID documents is a problem that can arise in various applications. A challenging instance of the problem arises when the face image on the ID document is from early adolescence and the live image is from later adolescence. We explore this problem using a private dataset called Chilean Young Adult (CHIYA) dataset, where we match live face images taken at age 18-19 to face images on scanned ID documents created at ages 9 to 18. State-of-the-art deep learning face matchers (e.g., ArcFace) have relatively poor accuracy for document-to-selfie face matching. To achieve higher accuracy, we fine-tune the best available open-source model with triplet loss for a few-shot learning. Experiments show that our approach achieves higher accuracy than the DocFace+ model recently developed for this problem. Our fine-tuned model was able to improve the true acceptance rate for the most difficult (largest age span) subset from 62.92% to 96.67% at a false acceptance rate of 0.01%. Our fine-tuned model is available for use by other researchers.

Albiero, Vítor↗

DOE OSTI2025

End-to-end protocol for high-quality quantum approximate optimization algorithm parameters with few shots

The quantum approximate optimization algorithm (QAOA) is a quantum heuristic for combinatorial optimization that has been demonstrated to scale better than state-of-the-art classical solvers for some problems. For a given problem instance, QAOA performance depends crucially on the choice of the parameters. While average-case optimal parameters are available in many cases, meaningful performance gains can be obtained by fine-tuning these parameters for a given instance. This task is especially challenging, however, when the number of circuit executions (shots) is limited. In this work, we develop an end-to-end protocol that combines multiple parameter settings and fine-tuning techniques. We use large-scale numerical experiments to optimize the protocol for the shot-limited setting and observe that optimizers with the simplest internal model (linear) perform best. We implement the optimized pipeline on a trapped-ion processor using up to 32 qubits and 5 QAOA layers, and we demonstrate that the pipeline is robust to small amounts of hardware noise. To the best of our knowledge, these are the largest demonstrations of QAOA parameter fine-tuning on a trapped-ion processor in terms of two-qubit gate count.

quantum algorithms & computation↗

DOE OSTI2025

Text Mining for Process–Structure–Properties Relationships in Metals

With the advent of large language models (LLMs), the vast unstructured text within millions of academic papers is increasingly accessible for materials discovery—although significant challenges remain. While LLMs offer promising few- and zero-shot learning capabilities, particularly valuable in the materials domain where expert annotations are scarce, general-purpose LLMs often fail to address key materials-specific queries without further adaptation. To bridge this gap, fine-tuning LLMs on human-labeled data is essential for effective structured knowledge extraction (Liu in The Importance of Human-Labeled Data in the Era of LLMs, 2023). Here, in this study, we introduce a novel annotation schema designed to extract generic process–structure–properties relationships from scientific literature. We demonstrate the utility of this approach using a dataset of 128 abstracts, with annotations drawn from two distinct domains: high-temperature materials (Domain I) and uncertainty quantification in simulating materials microstructure (Domain II). Initially, we developed a conditional random field (CRF) model based on MatBERT—a domain-specific BERT variant—and evaluated its performance on Domain I. Subsequently, we compared this model with a fine-tuned LLM (GPT-4o from OpenAI) under identical conditions. Our results indicate that fine-tuning LLMs can significantly improve entity extraction performance over the BERT-CRF baseline on Domain I. However, when additional examples from Domain II were incorporated, the performance of the BERT-CRF model became comparable to that of the GPT-4o model. These findings underscore the potential of our schema for structured knowledge extraction and highlight the complementary strengths of both modeling approaches.

Materials science↗

DOE OSTI2024

Supersymmetry with scalar sequestering

Supersymmetric models with a strongly interacting superconformal hidden sector (HS) may drive soft supersymmetry (SUSY) breaking scalar masses, bilinear soft term B μ and Higgs combinations m H u , d 2 + μ 2 to small values at some intermediate scale, leading to unique sparticle mass spectra along with possibly diminished fine-tuning in spite of a large superpotential μ parameter. We set up a computer code to calculate such spectra, which are then susceptible to a variety of constraints: (1) possible charge-or-color breaking (CCB) minima in the scalar potential, (2) unbounded from below (UFB) scalar potential, (3) improper electroweak symmetry breaking, (4) a charged or sneutrino lightest SUSY particle (LSP), (5) generating m h ∼ 125 GeV , (6) consistency with LHC sparticle mass limits, and (7) naturalness. We find this bevy of constraints leaves little or no viable parameter space for the case where hidden sector dynamics dominates minimal supersymmetric standard model (MSSM) running, even for the case of nonuniversal gaugino masses. For the case with moderate HS running with comparable MSSM running, and with universal gaugino masses, then the fine-tuning is ameliorated, but nonetheless remains high. Viable spectra with moderate HS running and with low fine-tuning and large μ can be found for nonuniversal gaugino masses. Published by the American Physical Society 2024

Astronomy & Astrophysics↗

DOE OSTI2024

Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models

Abstract We propose masked particle modeling (MPM) as a self-supervised method for learning generic, transferable, and reusable representations on unordered sets of inputs for use in high energy physics (HEP) scientific data. This work provides a novel scheme to perform masked modeling based pre-training to learn permutation invariant functions on sets. More generally, this work provides a step towards building large foundation models for HEP that can be generically pre-trained with self-supervised learning and later fine-tuned for a variety of down-stream tasks. In MPM, particles in a set are masked and the training objective is to recover their identity, as defined by a discretized token representation of a pre-trained vector quantized variational autoencoder. We study the efficacy of the method in samples of high energy jets at collider physics experiments, including studies on the impact of discretization, permutation invariance, and ordering. We also study the fine-tuning capability of the model, showing that it can be adapted to tasks such as supervised and weakly supervised jet classification, and that the model can transfer efficiently with small fine-tuning data sets to new classes and new data domains.

Heinrich, Lukas (ORCID:0000000240487584)↗

DOE OSTI2020

How Low Can You Go? Using Synthetic 3D Imagery to Drastically Reduce Real-World Training Data for Object Detection

Deep convolutional neural networks (DCNNs) currently provide state-of-the-art performance on image classification and object detection tasks, and there are many global security mission areas where such models could be extremely useful. Crucially, the success of these models is driven in large part by the widespread availability of high-quality open source data sets such as Image Net, Common Objects in Context (COCO), and KITTI, which contain millions of images with thousands of unique labels. However, global security relevant objects-of-interest can be difficult to obtain: relevant events are low frequency and high consequence; the content of relevant images is sensitive; and adversaries and proliferators seek to obscure their activities. For these cases where exemplar data is hard to come-by, even fine-tuning an existing model with available data can be effectively impossible. Recent work demonstrated that models can be trained using a combination of real-world and synthetic images generated from 3D representations; that such models can exceed the performance of models trained using real-world data alone; and that the generated images need not be perfectly realistic (Tremblay, et al., 2018). However, this approach still required hundreds to thousands of real-world images for training and fine tuning, which for sparse, global security-relevant datasets can be an unrealistic hurdle. In this research, we validate the performance and behavior of DCNN models as we drive the number of real-world images used for training object detection tasks down to a minimal set. We perform multiple experiments to identify the best approach to train DCNNs from an extremely small set of real-world images. In doing so, we: Develop state-of-the-art, parameterized 3D models based on real-world images and sample from their parameters to increase the variance in synthetic image training data; Use machine learning explainability techniques to highlight and correct through targeted training the biases that result from training using completely synthetic images; and Validate our results by comparing the performance of the models trained on synthetic data to one another, and to a control model created by fine-tuning an existing ImageNet-trained model with a limited number (hundreds) of real-world images.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Review of Grey Box/Black Box Data Contamination Metrics on Open and Commercial Models

Dataset contamination is a problem where benchmarks and tasks used to evaluate the capabilities of Large Language Models (LLMs) have been incorporated into the training dataset of the models. This gives a false sense of performance that can overestimate how these models will function on truly unseen data. This problem becomes worse with commercial LLMs with larger and non-accessible training data, so techniques have been developed to try to measure the degree to which a model is contaminated with a benchmark’s data. To understand the effectiveness of these techniques, particularly when evaluating contamination on coding tasks, we review trends and categorize techniques by the degree of access to the model that is required. The research literature on this topic has reported mixed effectiveness of these techniques, so we select a set of black box (text access only) and grey box (access to model loss/probabilities required) techniques and apply them to both commercial and non-commercial models. We implement these metrics as part of a framework to test the contamination of Python code in LLMs to see to what extent we can replicate the effectiveness (or ineffectiveness) of these contamination detection techniques. Though we find mixed results in the capabilities of these metrics to identify contamination, we do observe evidence that they can identify contamination (broadly) in fine-tuned models when both a baseline and fine-tuned model is present. Additionally, similarity metrics were able to identify between contaminated and uncontaminated data even in situations where the data is distributionally similar (e.g., drawn from the same set of code projects).

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

chatHPC: Empowering HPC users with large language models

The ever-growing number of pre-trained large language models (LLMs) across scientific domains presents a challenge for application developers. While these models offer vast potential, fine-tuning them with custom data, aligning them for specific tasks, and evaluating their performance remain crucial steps for effective utilization. However, applying these techniques to models with tens of billions of parameters can take days or even weeks on modern workstations, making the cumulative cost of model comparison and evaluation a significant barrier to LLM-based application development. To address this challenge, we introduce an end-to-end pipeline specifically designed for building conversational and programmable AI agents on high performance computing (HPC) platforms. Our comprehensive pipeline encompasses: model pre-training, fine-tuning, web and API service deployment, along with crucial evaluations for lexical coherence, semantic accuracy, hallucination detection, and privacy considerations. Here, we demonstrate our pipeline through the development of chatHPC, a chatbot for HPC question answering and script generation. Leveraging our scalable pipeline, we achieve end-to-end LLM alignment in under an hour on the Frontier supercomputer. We propose a novel self-improved, self-instruction method for instruction set generation, investigate scaling and fine-tuning strategies, and conduct a systematic evaluation of model performance. The established practices within chatHPC will serve as a valuable guidance for future LLM-based application development on HPC platforms.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Bell inequalities in 2-2 scattering

We consider Bell inequalities in 2-2 scattering of photons, gravitons, fermions, and pions. We choose measurement settings that give maximum Bell violation for maximally entangled states and calculate the relevant Bell inequalities for these processes. For photon scattering at low energies, quantum electrodynamics exhibits Bell violation for all scattering angles except for a small transverse region. This leads to a fine-tuning problem. Incorporating a light axion/axionlike particle removes the fine-tuning problem and constrains the axion-coupling–axion-mass parameters. Allowing for graviton exchange and demanding Bell violation in photon scattering, we find that the weak-gravity conjecture is satisfied. Quantum gravity effects on axion coupling is discussed. For 2-2 graviton scattering, we find that CEMZ bounds allow for, at most, small Bell violations. Restriction on the Weinberg angle is found by demanding Bell violation in Bhabha scattering. We use recent $S$-matrix bootstrap data for pions and photons to study the Bell parameter in the space of allowed $S$-matrices. In the photon case, we study the Bell parameters as a function of energy and find support for the effective field theory observations. We discuss Bell parameter for pion $S$-matrices, which are qutrits. For pions, we find that there is a minimization of a suitable Bell parameter for $S$-matrices which exhibit Regge behavior.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2020

Survey of Image Denoising Methods for Medical Image Classification

Medical imaging devices, such as X-ray machines, inherently produce images that suffer from visual noise. Our objectives were to (i.) determine the effect of image denoising on a medical image classification task, and (ii.) determine if there exists a correlation between image denoising performance and medical image classification performance. We performed the medical image classification task on chest X-rays using the DenseNet-121 convolutional neural network (CNN) and used the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) metrics as the image denoising performance measures. We first found that different denoising methods can make a statistically significant difference in classification performance for select labels. We also found that denoising methods affect fine-tuned models more than randomly-initialized models and that fine-tuned models have significantly higher and more uniform performance than randomly-initialized models. Lastly, we found that there is no significant correlation between PSNR and SSIM values and classification performance for our task.

Michael, Peter↗

DOE OSTI2023

Sr(Ag 1-x Li x ) 2 Se 2 and [Sr 3 Se 2 ][(Ag 1-x Li x ) 2 Se 2 ] Tunable Direct Band Gap Semiconductors

Synthesizing solids in molten fluxes enables the rapid diffusion of soluble species at temperatures lower than in solid-state reactions, leading to crystal formation of kinetically stable compounds. Here, in this study, we demonstrate the effectiveness of mixed hydroxide and halide fluxes in synthesizing complex Sr/Ag/Se in mixed LiOH/LiCl. We have accessed a series of two-dimensional Sr(Ag 1-x Li x ) 2 Se 2 layered phases. With increased LiOH/LiCl ratio or reaction temperature, Li partially substituted Ag to form solid solutions of Sr(Ag 1-x Li x ) 2 Se 2 with x up to 0.45. In addition, a new type of intergrowth compound [Sr 3 Se 2 ][(Ag 1-x Li x ) 2 Se 2 ] was synthesized upon further reaction of Sr(Ag 1-x Li x ) 2 Se 2 with SrSe. Both Sr(Ag 1-x Li x ) 2 Se 2 and [Sr 3 Se 2 ][(Ag 1-x Li x ) 2 Se 2 ] exhibit a direct band gap, which increases with increasing Li substitution (x). Therefore, the band gap of Sr(Ag 1-x Li x ) 2 Se 2 can be precisely tuned via fine-tuning x that is controlled by only the flux ratio and temperature.

2D materials↗

DOE OSTI2025

OmniXAS: A universal deep-learning framework for materials x-ray absorption spectra

X-ray absorption spectroscopy (XAS) is a powerful characterization technique for probing the local chemical environment of absorbing atoms. However, analyzing XAS data presents significant challenges, often requiring extensive, computationally intensive simulations, as well as significant domain expertise. These limitations hinder the development of fast, robust XAS analysis pipelines that are essential in high-throughput studies and for autonomous experimentation. Here, we address these challenges with OmniXAS, a framework that contains a suite of transfer learning approaches for XAS prediction, each uniquely contributing to improved accuracy and efficiency, as demonstrated on the K-edge spectra database covering eight 3⁢d transition metals (Ti–Cu). The OmniXAS framework is built upon three distinct strategies. First, we use M3GNet [Nat. Comput. Sci. 2, 718 (2022)] to derive latent representations of the local chemical environment of absorption sites as input for XAS prediction, achieving significant improvements over conventional featurization techniques. Second, we employ a hierarchical transfer learning strategy, training a universal multitask model across elements before fine-tuning for element-specific predictions. Models based on this cascaded approach after elementwise fine-tuning outperform element-specific models by up to 69%. Third, we implement cross-fidelity transfer learning, adapting a universal model to predict spectra generated by simulation of a different fidelity with a much higher computational cost. This approach improves prediction accuracy by up to 11% over models trained on the target fidelity alone. Our approach significantly boosts the throughput of XAS modeling by orders of magnitude as compared to first-principles simulations and is extendable to XAS prediction for a broader range of elements. The proposed transfer learning framework is generalizable to enhance deep-learning models that target other properties in materials research.

36 MATERIALS SCIENCE↗

DOE OSTI2026

Augmenting Molecular Graphs with Geometries via Machine Learning Interatomic Potentials

Accurate molecular property predictions require 3D geometries, which are typically obtained using expensive methods such as density functional theory (DFT). Here, we attempt to obtain molecular geometries by relying solely on machine learning interatomic potential (MLIP) models. To this end, we first curate a large-scale molecular relaxation dataset comprising 3.5 million molecules and 300 million snapshots. Then MLIP pre-trained models are trained with supervised learning to predict energy and forces given 3D molecular structures. Once trained, we show that the pre-trained models can be used in different ways to obtain geometries either explicitly or implicitly. First, it can be used to obtain approximate low-energy 3D geometries via geometry optimization. While these geometries do not consistently reach DFT-level chemical accuracy or convergence, they can still improve downstream performance compared to non-relaxed structures. To mitigate potential biases and enhance downstream predictions, we introduce geometry fine-tuning based on the relaxed 3D geometries. Second, the pre-trained models can be directly fine-tuned for property prediction when ground truth 3D geometries are available. Our results demonstrate that MLIP pre-trained models trained on relaxation data can learn transferable molecular representations to improve downstream molecular property prediction and can provide practically valuable but approximate molecular geometries that benefit property predictions. Our code is publicly available at: https://github.com/divelab/AIRS/.

Fu, Cong [Texas A & M Univ., College Station, TX (↗