Search NASA⌕ Search

SEARCH · Search NASA

Results for “fine tuning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Semantic Search with Sentence-BERT for Design Information Retrieval

Managing and referencing design knowledge is a critical activity in the design process. However, reliably retrieving useful knowledge can be a frustrating experience for users of knowledge management systems due to inherent limitations of standard keyword-based searches. In this research, we consider the task of retrieving relevant lessons learned from the NASA Lessons Learned Information System (LLIS). To this end, we apply a state-of-the-art natural language processing (NLP) technique for information retrieval (IR): semantic search with sentence-BERT, which is a modification of a Bidirectional Encoder Representations from Transformers (BERT) model that uses siamese and triplet network architectures to obtain semantically meaningful sentence embeddings. While the pre-trained sBERT model performs well out-of-the-box, we further fine-tune the model on data from the LLIS so that it learns on design engineering-relevant vocabulary. We quantify the improvement in query results using both standard sBERT and fine-tuned sBERT over a keyword search. Our use case throughout the paper is to use queries related to specific requirements from a NASA project. Fine tuning the sBERT model on LLIS data yields a mean average precision (MAP) of 0.807 on queries based on information needs from a real NASA project. Results indicate that applying state-of-the-art natural language processing techniques, especially when finetuned using engineering data, to design information retrieval tasks shows significant promise in modernizing design knowledge management systems.

Hannah S. Walsh↗
NASA NTRSDate not supplied

Semantic Search with Sentence-BERT for Design Information Retrieval

Managing and referencing design knowledge is a critical activity in the design process. However, reliably retrieving useful knowledge can be a frustrating experience for users of knowledge management systems due to inherent limitations of standard keyword-based searches. In this research, we consider the task of retrieving relevant lessons learned from the NASA Lessons Learned Information System (LLIS). To this end, we apply a state-of-the-art natural language processing (NLP) technique for information retrieval: semantic search with sentence-BERT, which is a modification of a Bidirectional Encoder Representations from Transformers (BERT) model that uses siamese and triplet network architectures to obtain semantically meaningful sentence embeddings. While the pretrained sBERT model shows excellent out-of-the-box performance, we further fine-tune the model on data from the LLIS so that it learns on design engineering-relevant vocabulary. We quantify the improvement in query results using both standard sBERT and fine-tuned sBERT over the LLIS’s built-in keyword search. Additionally, we demonstrate a use case for the query system by searching for lessons learned relevant to specific requirements from a NASA project as part of a broader knowledge management and retrieval system. Results indicate that applying state-of-the-art natural language processing techniques, especially when fine-tuned using engineering data, to design information retrieval tasks shows significant promise in modernizing design knowledge management systems.

Hannah S. Walsh↗
DOE OSTI2021

Identity Document to Selfie Face Matching Across Adolescence

Matching live images (“selfies”) to images from ID documents is a problem that can arise in various applications. A challenging instance of the problem arises when the face image on the ID document is from early adolescence and the live image is from later adolescence. We explore this problem using a private dataset called Chilean Young Adult (CHIYA) dataset, where we match live face images taken at age 18-19 to face images on scanned ID documents created at ages 9 to 18. State-of-the-art deep learning face matchers (e.g., ArcFace) have relatively poor accuracy for document-to-selfie face matching. To achieve higher accuracy, we fine-tune the best available open-source model with triplet loss for a few-shot learning. Experiments show that our approach achieves higher accuracy than the DocFace+ model recently developed for this problem. Our fine-tuned model was able to improve the true acceptance rate for the most difficult (largest age span) subset from 62.92% to 96.67% at a false acceptance rate of 0.01%. Our fine-tuned model is available for use by other researchers.

Albiero, Vítor↗
DOE OSTI2025

End-to-end protocol for high-quality quantum approximate optimization algorithm parameters with few shots

The quantum approximate optimization algorithm (QAOA) is a quantum heuristic for combinatorial optimization that has been demonstrated to scale better than state-of-the-art classical solvers for some problems. For a given problem instance, QAOA performance depends crucially on the choice of the parameters. While average-case optimal parameters are available in many cases, meaningful performance gains can be obtained by fine-tuning these parameters for a given instance. This task is especially challenging, however, when the number of circuit executions (shots) is limited. In this work, we develop an end-to-end protocol that combines multiple parameter settings and fine-tuning techniques. We use large-scale numerical experiments to optimize the protocol for the shot-limited setting and observe that optimizers with the simplest internal model (linear) perform best. We implement the optimized pipeline on a trapped-ion processor using up to 32 qubits and 5 QAOA layers, and we demonstrate that the pipeline is robust to small amounts of hardware noise. To the best of our knowledge, these are the largest demonstrations of QAOA parameter fine-tuning on a trapped-ion processor in terms of two-qubit gate count.

quantum algorithms & computation↗
DOE OSTI2025

Text Mining for Process–Structure–Properties Relationships in Metals

With the advent of large language models (LLMs), the vast unstructured text within millions of academic papers is increasingly accessible for materials discovery—although significant challenges remain. While LLMs offer promising few- and zero-shot learning capabilities, particularly valuable in the materials domain where expert annotations are scarce, general-purpose LLMs often fail to address key materials-specific queries without further adaptation. To bridge this gap, fine-tuning LLMs on human-labeled data is essential for effective structured knowledge extraction (Liu in The Importance of Human-Labeled Data in the Era of LLMs, 2023). Here, in this study, we introduce a novel annotation schema designed to extract generic process–structure–properties relationships from scientific literature. We demonstrate the utility of this approach using a dataset of 128 abstracts, with annotations drawn from two distinct domains: high-temperature materials (Domain I) and uncertainty quantification in simulating materials microstructure (Domain II). Initially, we developed a conditional random field (CRF) model based on MatBERT—a domain-specific BERT variant—and evaluated its performance on Domain I. Subsequently, we compared this model with a fine-tuned LLM (GPT-4o from OpenAI) under identical conditions. Our results indicate that fine-tuning LLMs can significantly improve entity extraction performance over the BERT-CRF baseline on Domain I. However, when additional examples from Domain II were incorporated, the performance of the BERT-CRF model became comparable to that of the GPT-4o model. These findings underscore the potential of our schema for structured knowledge extraction and highlight the complementary strengths of both modeling approaches.

Materials science↗
DOE OSTI2024

Supersymmetry with scalar sequestering

Supersymmetric models with a strongly interacting superconformal hidden sector (HS) may drive soft supersymmetry (SUSY) breaking scalar masses, bilinear soft term B μ and Higgs combinations m H u , d 2 + μ 2 to small values at some intermediate scale, leading to unique sparticle mass spectra along with possibly diminished fine-tuning in spite of a large superpotential μ parameter. We set up a computer code to calculate such spectra, which are then susceptible to a variety of constraints: (1) possible charge-or-color breaking (CCB) minima in the scalar potential, (2) unbounded from below (UFB) scalar potential, (3) improper electroweak symmetry breaking, (4) a charged or sneutrino lightest SUSY particle (LSP), (5) generating m h ∼ 125 GeV , (6) consistency with LHC sparticle mass limits, and (7) naturalness. We find this bevy of constraints leaves little or no viable parameter space for the case where hidden sector dynamics dominates minimal supersymmetric standard model (MSSM) running, even for the case of nonuniversal gaugino masses. For the case with moderate HS running with comparable MSSM running, and with universal gaugino masses, then the fine-tuning is ameliorated, but nonetheless remains high. Viable spectra with moderate HS running and with low fine-tuning and large μ can be found for nonuniversal gaugino masses. Published by the American Physical Society 2024

Astronomy & Astrophysics↗
DOE OSTI2024

Masked Particle Modeling on Sets: Towards Self-Supervised High Energy Physics Foundation Models

Abstract We propose masked particle modeling (MPM) as a self-supervised method for learning generic, transferable, and reusable representations on unordered sets of inputs for use in high energy physics (HEP) scientific data. This work provides a novel scheme to perform masked modeling based pre-training to learn permutation invariant functions on sets. More generally, this work provides a step towards building large foundation models for HEP that can be generically pre-trained with self-supervised learning and later fine-tuned for a variety of down-stream tasks. In MPM, particles in a set are masked and the training objective is to recover their identity, as defined by a discretized token representation of a pre-trained vector quantized variational autoencoder. We study the efficacy of the method in samples of high energy jets at collider physics experiments, including studies on the impact of discretization, permutation invariance, and ordering. We also study the fine-tuning capability of the model, showing that it can be adapted to tasks such as supervised and weakly supervised jet classification, and that the model can transfer efficiently with small fine-tuning data sets to new classes and new data domains.

Heinrich, Lukas (ORCID:0000000240487584)↗
NASA NTRSDate not supplied

AI Foundation Models for Science: An Open Collaborative Initiative

Foundation Models (FMs), AI models designed to replace task-specific models, are increasingly being recognized for their versatility across numerous downstream applications. These models, trained using self-supervised techniques on any type of sequence data, circumvent the need for large annotated datasets, a major bottleneck in traditional AI model development. FMs can be applied to downstream tasks using few-shot learning and fine-tuning, significantly reducing the need for large labeled training datasets and computational resources. However, the development of FMs requires substantial resources, including access to data and compute power, expertise in the latest models, and specialized scientific knowledge for systematic evaluation. It is challenging for a single group to possess all these capabilities. To address this, NASA IMPACT has initiated an open collaborative effort, leveraging partnerships with the private sector and other groups within and outside NASA, to jointly build FMs. The overarching goal is to develop a consistent and collaborative approach to building FMs for high-value science datasets. This initiative has fostered collaboration within NASA and with external partners, including IBM Research, Clark University, DOE’s ORNL, ESA, and USGS. The effort focuses on identifying key datasets with a wide range of downstream applications, pretraining and building FMs using modified transformer architectures, evaluating compute infrastructure needs, and sharing models, pretraining and fine-tuning code, and data with the community. Furthermore, it aims to train the Earth science community to fine-tune these models for various downstream applications. Our initial effort resulted in the creation of a 100 million parameter HLS Geospatial Model within six months, which was released on HuggingFace. We are now expanding our scope to include data from weather and climate models and investigating multimodal models. We invite those interested in participating in this effort to join us by sharing their use cases, expertise, or data.

Rahul Ramachandran↗
DOE OSTI2020

How Low Can You Go? Using Synthetic 3D Imagery to Drastically Reduce Real-World Training Data for Object Detection

Deep convolutional neural networks (DCNNs) currently provide state-of-the-art performance on image classification and object detection tasks, and there are many global security mission areas where such models could be extremely useful. Crucially, the success of these models is driven in large part by the widespread availability of high-quality open source data sets such as Image Net, Common Objects in Context (COCO), and KITTI, which contain millions of images with thousands of unique labels. However, global security relevant objects-of-interest can be difficult to obtain: relevant events are low frequency and high consequence; the content of relevant images is sensitive; and adversaries and proliferators seek to obscure their activities. For these cases where exemplar data is hard to come-by, even fine-tuning an existing model with available data can be effectively impossible. Recent work demonstrated that models can be trained using a combination of real-world and synthetic images generated from 3D representations; that such models can exceed the performance of models trained using real-world data alone; and that the generated images need not be perfectly realistic (Tremblay, et al., 2018). However, this approach still required hundreds to thousands of real-world images for training and fine tuning, which for sparse, global security-relevant datasets can be an unrealistic hurdle. In this research, we validate the performance and behavior of DCNN models as we drive the number of real-world images used for training object detection tasks down to a minimal set. We perform multiple experiments to identify the best approach to train DCNNs from an extremely small set of real-world images. In doing so, we: Develop state-of-the-art, parameterized 3D models based on real-world images and sample from their parameters to increase the variance in synthetic image training data; Use machine learning explainability techniques to highlight and correct through targeted training the biases that result from training using completely synthetic images; and Validate our results by comparing the performance of the models trained on synthetic data to one another, and to a control model created by fine-tuning an existing ImageNet-trained model with a limited number (hundreds) of real-world images.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Review of Grey Box/Black Box Data Contamination Metrics on Open and Commercial Models

Dataset contamination is a problem where benchmarks and tasks used to evaluate the capabilities of Large Language Models (LLMs) have been incorporated into the training dataset of the models. This gives a false sense of performance that can overestimate how these models will function on truly unseen data. This problem becomes worse with commercial LLMs with larger and non-accessible training data, so techniques have been developed to try to measure the degree to which a model is contaminated with a benchmark’s data. To understand the effectiveness of these techniques, particularly when evaluating contamination on coding tasks, we review trends and categorize techniques by the degree of access to the model that is required. The research literature on this topic has reported mixed effectiveness of these techniques, so we select a set of black box (text access only) and grey box (access to model loss/probabilities required) techniques and apply them to both commercial and non-commercial models. We implement these metrics as part of a framework to test the contamination of Python code in LLMs to see to what extent we can replicate the effectiveness (or ineffectiveness) of these contamination detection techniques. Though we find mixed results in the capabilities of these metrics to identify contamination, we do observe evidence that they can identify contamination (broadly) in fine-tuned models when both a baseline and fine-tuned model is present. Additionally, similarity metrics were able to identify between contaminated and uncontaminated data even in situations where the data is distributionally similar (e.g., drawn from the same set of code projects).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

chatHPC: Empowering HPC users with large language models

The ever-growing number of pre-trained large language models (LLMs) across scientific domains presents a challenge for application developers. While these models offer vast potential, fine-tuning them with custom data, aligning them for specific tasks, and evaluating their performance remain crucial steps for effective utilization. However, applying these techniques to models with tens of billions of parameters can take days or even weeks on modern workstations, making the cumulative cost of model comparison and evaluation a significant barrier to LLM-based application development. To address this challenge, we introduce an end-to-end pipeline specifically designed for building conversational and programmable AI agents on high performance computing (HPC) platforms. Our comprehensive pipeline encompasses: model pre-training, fine-tuning, web and API service deployment, along with crucial evaluations for lexical coherence, semantic accuracy, hallucination detection, and privacy considerations. Here, we demonstrate our pipeline through the development of chatHPC, a chatbot for HPC question answering and script generation. Leveraging our scalable pipeline, we achieve end-to-end LLM alignment in under an hour on the Frontier supercomputer. We propose a novel self-improved, self-instruction method for instruction set generation, investigate scaling and fine-tuning strategies, and conduct a systematic evaluation of model performance. The established practices within chatHPC will serve as a valuable guidance for future LLM-based application development on HPC platforms.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2006

Multichannel X-Band Dielectric-Resonator Oscillator

A multichannel dielectric-resonator oscillator (DRO), built as a prototype of a local oscillator for an X-band transmitter or receiver, is capable of being electrically tuned among and within 26 adjacent frequency channels, each 1.16 MHz wide, in a band ranging from 7,040 to 7,070 GHz. The tunability of this oscillator is what sets it apart from other DROs, making it possible to use mass-produced oscillator units of identical design in diverse X-band applications in which there are requirements to use different fixed frequencies or to switch among frequency channels. The oscillator (see figure) includes a custom-designed voltage-controlled-oscillator (VCO) monolithic microwave integrated circuit (MMIC), a dielectric resonator disk (puck), and two varactor-coupling circuits, all laid out on a 25-mil (0.635-mm)-thick alumina substrate having a length and width of 17.8 mm. The resonator disk has a diameter of 8.89 mm and a thickness of 4.01 mm. The oscillator is mounted in an 8.9-mm-deep cavity in a metal housing. The VCO MMIC incorporates a negative- resistance oscillator amplifier along with a buffer amplifier. The resonator disk is coupled to a microstrip transmission line connected to the negative-resistance port of the VCO MMIC. The two varactor-coupling circuits include microstrip lines, laid out orthogonally to each other, for coupling with the resonator disk. Each varactor microstrip line is DC-coupled to an external port via a microwave choke. One varactor is used for coarse tuning to select a channel; the other varactor is used (1) for fine tuning across the 1.16-MHz width of each channel and (2) as a feedback port for a phase-lock loop. The resonator disk is positioned to obtain (1) the most desirable bandwidth, (2) relatively tight coupling with the microstrip connected to the coarse-tuning varactor, and (3) relatively loose coupling with the microstrip connected to the fine-tuning varactor. Measurements of performance showed that the oscillator can be switched among any of the 26 channels and can be phase-locked to a nominal frequency in any channel. The degree of nonlinearity of tuning was found not to exceed 2.5 percent. The tuning sensitivity was found to be 6.15 MHz/V at a bias offset of -2 V on the phase-lock-loop varactor. The phase noise of the oscillator in free-running operation was found to be -107 dBc/Hz (where dBc signifies decibels relative to the carrier signal) at 100 kHz away from the carrier frequency.

Mysoor, Narayan↗
DOE OSTI2023

Bell inequalities in 2-2 scattering

We consider Bell inequalities in 2-2 scattering of photons, gravitons, fermions, and pions. We choose measurement settings that give maximum Bell violation for maximally entangled states and calculate the relevant Bell inequalities for these processes. For photon scattering at low energies, quantum electrodynamics exhibits Bell violation for all scattering angles except for a small transverse region. This leads to a fine-tuning problem. Incorporating a light axion/axionlike particle removes the fine-tuning problem and constrains the axion-coupling–axion-mass parameters. Allowing for graviton exchange and demanding Bell violation in photon scattering, we find that the weak-gravity conjecture is satisfied. Quantum gravity effects on axion coupling is discussed. For 2-2 graviton scattering, we find that CEMZ bounds allow for, at most, small Bell violations. Restriction on the Weinberg angle is found by demanding Bell violation in Bhabha scattering. We use recent $S$-matrix bootstrap data for pions and photons to study the Bell parameter in the space of allowed $S$-matrices. In the photon case, we study the Bell parameters as a function of energy and find support for the effective field theory observations. We discuss Bell parameter for pion $S$-matrices, which are qutrits. For pions, we find that there is a minimization of a suitable Bell parameter for $S$-matrices which exhibit Regge behavior.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Survey of Image Denoising Methods for Medical Image Classification

Medical imaging devices, such as X-ray machines, inherently produce images that suffer from visual noise. Our objectives were to (i.) determine the effect of image denoising on a medical image classification task, and (ii.) determine if there exists a correlation between image denoising performance and medical image classification performance. We performed the medical image classification task on chest X-rays using the DenseNet-121 convolutional neural network (CNN) and used the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) metrics as the image denoising performance measures. We first found that different denoising methods can make a statistically significant difference in classification performance for select labels. We also found that denoising methods affect fine-tuned models more than randomly-initialized models and that fine-tuned models have significantly higher and more uniform performance than randomly-initialized models. Lastly, we found that there is no significant correlation between PSNR and SSIM values and classification performance for our task.

Michael, Peter↗
DOE OSTI2023

Sr(Ag 1-x Li x ) 2 Se 2 and [Sr 3 Se 2 ][(Ag 1-x Li x ) 2 Se 2 ] Tunable Direct Band Gap Semiconductors

Synthesizing solids in molten fluxes enables the rapid diffusion of soluble species at temperatures lower than in solid-state reactions, leading to crystal formation of kinetically stable compounds. Here, in this study, we demonstrate the effectiveness of mixed hydroxide and halide fluxes in synthesizing complex Sr/Ag/Se in mixed LiOH/LiCl. We have accessed a series of two-dimensional Sr(Ag 1-x Li x ) 2 Se 2 layered phases. With increased LiOH/LiCl ratio or reaction temperature, Li partially substituted Ag to form solid solutions of Sr(Ag 1-x Li x ) 2 Se 2 with x up to 0.45. In addition, a new type of intergrowth compound [Sr 3 Se 2 ][(Ag 1-x Li x ) 2 Se 2 ] was synthesized upon further reaction of Sr(Ag 1-x Li x ) 2 Se 2 with SrSe. Both Sr(Ag 1-x Li x ) 2 Se 2 and [Sr 3 Se 2 ][(Ag 1-x Li x ) 2 Se 2 ] exhibit a direct band gap, which increases with increasing Li substitution (x). Therefore, the band gap of Sr(Ag 1-x Li x ) 2 Se 2 can be precisely tuned via fine-tuning x that is controlled by only the flux ratio and temperature.

2D materials↗
DOE OSTI2025

OmniXAS: A universal deep-learning framework for materials x-ray absorption spectra

X-ray absorption spectroscopy (XAS) is a powerful characterization technique for probing the local chemical environment of absorbing atoms. However, analyzing XAS data presents significant challenges, often requiring extensive, computationally intensive simulations, as well as significant domain expertise. These limitations hinder the development of fast, robust XAS analysis pipelines that are essential in high-throughput studies and for autonomous experimentation. Here, we address these challenges with OmniXAS, a framework that contains a suite of transfer learning approaches for XAS prediction, each uniquely contributing to improved accuracy and efficiency, as demonstrated on the K-edge spectra database covering eight 3⁢d transition metals (Ti–Cu). The OmniXAS framework is built upon three distinct strategies. First, we use M3GNet [Nat. Comput. Sci. 2, 718 (2022)] to derive latent representations of the local chemical environment of absorption sites as input for XAS prediction, achieving significant improvements over conventional featurization techniques. Second, we employ a hierarchical transfer learning strategy, training a universal multitask model across elements before fine-tuning for element-specific predictions. Models based on this cascaded approach after elementwise fine-tuning outperform element-specific models by up to 69%. Third, we implement cross-fidelity transfer learning, adapting a universal model to predict spectra generated by simulation of a different fidelity with a much higher computational cost. This approach improves prediction accuracy by up to 11% over models trained on the target fidelity alone. Our approach significantly boosts the throughput of XAS modeling by orders of magnitude as compared to first-principles simulations and is extendable to XAS prediction for a broader range of elements. The proposed transfer learning framework is generalizable to enhance deep-learning models that target other properties in materials research.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Augmenting Molecular Graphs with Geometries via Machine Learning Interatomic Potentials

Accurate molecular property predictions require 3D geometries, which are typically obtained using expensive methods such as density functional theory (DFT). Here, we attempt to obtain molecular geometries by relying solely on machine learning interatomic potential (MLIP) models. To this end, we first curate a large-scale molecular relaxation dataset comprising 3.5 million molecules and 300 million snapshots. Then MLIP pre-trained models are trained with supervised learning to predict energy and forces given 3D molecular structures. Once trained, we show that the pre-trained models can be used in different ways to obtain geometries either explicitly or implicitly. First, it can be used to obtain approximate low-energy 3D geometries via geometry optimization. While these geometries do not consistently reach DFT-level chemical accuracy or convergence, they can still improve downstream performance compared to non-relaxed structures. To mitigate potential biases and enhance downstream predictions, we introduce geometry fine-tuning based on the relaxed 3D geometries. Second, the pre-trained models can be directly fine-tuned for property prediction when ground truth 3D geometries are available. Our results demonstrate that MLIP pre-trained models trained on relaxation data can learn transferable molecular representations to improve downstream molecular property prediction and can provide practically valuable but approximate molecular geometries that benefit property predictions. Our code is publicly available at: https://github.com/divelab/AIRS/.

Fu, Cong [Texas A & M Univ., College Station, TX (↗
DOE OSTI2023

Constraints on the scalar-field potential in warm inflation

In this work, we quantify the degree of fine-tuning required to achieve an observationally viable period of inflation in the strongly dissipative regime of warm inflation. The “fine-tuning” parameter λ is taken to be the ratio of the change in the height of the potential Δ V to the change in the scalar field (ΔΦ) 4 , i.e., the width of the potential, and therefore measures the requisite degree of flatness in the potential. The best motivated warm inflationary scenarios involve a dissipation rate of the kind Γ ∝ T c with c ≥ 0 , and for all such cases, the bounds on λ are tighter than those for standard cold inflation by at least 3 orders of magnitude. In other words, these models require an even flatter potential than standard inflation. On the other hand for the case of warm inflation with c < 0, we find that in a strongly dissipative regime the bound on λ can significantly weaken with respect to cold inflation. Thus, if a warm inflation model can be constructed in a strongly dissipative, negatively temperature-dependent regime, it accommodates steeper potentials otherwise ruled out in standard inflation.

79 ASTRONOMY AND ASTROPHYSICS↗