Search NASA⌕ Search

SEARCH · Search NASA

Results for “embedding model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

NEAR: Neural Embeddings for Amino acid Relationships

Protein language models (PLMs) have recently demonstrated potential to supplant classical protein database search methods based on sequence alignment, but are slower than common alignment-based tools and appear to be prone to a high rate of false labeling. Here, we present NEAR, a method based on neural representation learning that is designed to improve both speed and accuracy of search for likely homologs in a large protein sequence database. NEAR’s ResNet embedding model is trained using contrastive learning guided by trusted sequence alignments. It computes per-residue embeddings for target and query protein sequences, and identifies alignment candidates with a pipeline consisting of residue-level k-NN search and a simple neighbor aggregation scheme. Tests on a benchmark consisting of trusted remote homologs and randomly shuffled decoy sequences reveal that NEAR substantially improves accuracy relative to state-of-the-art PLMs, with lower memory requirements and faster embedding and search speed. While these results suggest that the NEAR model may be useful for standalone homology detection with increased sensitivity over standard alignment-based methods, in this manuscript we focus on a more straightforward analysis of the model’s value as a high-speed pre-filter for sensitive annotation. In that context, NEAR is at least 5x faster than the pre-filter currently used in the widely-used profile hidden Markov model (pHMM) search tool HMMER3, and also outperforms the pre-filter used in our fast pHMM tool, nail.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Thermal Modeling and Limitations for Power Electronics Embedded in Medium-Voltage Cables

As next-generation energy technologies gain traction and power demand increases, the existing electrical infrastructure faces significant stress, prompting innovative solutions to enhance the grid's capacity and lifespan. This work explores the possibility of embedding medium-voltage (MV) power electronics directly inline with the cable, and the resulting thermal challenges. Since the majority of power distribution cables installed in the U.S. are passively cooled, the work focuses primarily on passive cooling, with an emphasis on the limitations of axial heat spreading within the cable. To date, literature on axial spreading of high incident heat loads on cables and cable environments is limited, typically reporting cases with <10 W of incident heat load. This work will explore the considerations, limits, and tradeoffs of cable-embedded heat loads significantly larger than the cable losses. Both external and internal effects are modeled analytically in nondimensional terms via a Biot number analysis, allowing fundamental limits and tradeoffs to be derived. The work culminates in the design and experimental validation of a cable-embedded thermal system capable of passively dissipating 300 W of heat from a coaxial SiC mosfet switch module over a length of 20 cm, thus validating the possibility of MV cable-embedded power electronics from a thermal standpoint.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Dispersion Loss Counteracts Embedding Condensation and Improves Generalization in Small Language Models

Large language models (LLMs) achieve remarkable performance through ever-increasing parameter counts, but scaling incurs steep computational costs. To better understand LLM scaling, we study representational differences between LLMs and their smaller counterparts, with the goal of replicating the representational qualities of larger models in smaller models. We observe a geometric phenomenon which we term embedding condensation, where token embeddings collapse into a narrow cone-like subspace in some language models. Through systematic analyses across multiple Transformer families, we show that small models such as GPT2 and Qwen3-0.6B exhibit severe condensation, whereas larger models such as GPT2-x1 and Qwen3-32B are more resistant to this phenomenon. Additional observations show that embedding condensation is not reliably mitigated by knowledge distillation from larger models. To fight against it, we formulate a dispersion loss that explicitly encourages embedding dispersion during training. Experiments demonstrate that it mitigates condensation, recovers dispersion patterns seen in larger models, and yields performance gains across 10 benchmarks. We believe this work offers a principled path toward improving smaller Transformers without additional parameters.

Xiao, Xi [ORNL] (ORCID:0009000009316982)↗
DOE OSTI2019

Rotationally invariant slave-boson and density matrix embedding theory: Unified framework and comparative study on the one-dimensional and two-dimensional Hubbard model [Rotationally invariant slave-boson and density matrix embedding theory: A unified framework and a comparative study on the 1D and 2D Hubbard Model]

We present detailed benchmark ground-state calculations of the one- and two-dimensional Hubbard model utilizing the cluster extensions of the rotationally invariant slave-boson mean-field theory and the density matrix embedding theory. Our analysis shows that the overall accuracy and the performance of these two methods are very similar. Furthermore, we propose a unified computational framework that allows us to implement both of these techniques on the same footing. In conclusion, this provides us with a different line of interpretation and paves the ways for developing systematically distinct generalizations of these complementary approaches.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2019

Expanding the computational frontier of multi-scale atmospheric simulation to advance understanding of low cloud / climate feedbacks: Final Technical Report

Marine boundary layer clouds are a challenge for climate models. They cover much of the oceans, but they are driven by small-scale turbulent eddies only a few hundred meters across, which in turn respond to the cloud formation. Global climate models have a grid spacing that is far too large to simulate such fluid motions, so they parameterize these cloud formation processes. Different climate modeling groups have designed different parameterizations in which the clouds turn out to be differently sensitive to a warming climate. This drives uncertainties in our best guess at the sensitivity of global warming to greenhouse gas increases. If these cloud-forming eddies could be directly simulated using the well-known equations of fluid motion, they would no longer need to be parameterized, removing a major source of climate modeling uncertainty. In this project, we overcame software engineering challenges to successfully implemented ‘ultraparameterization’ (UP), the first global model that does this, and we tested how well it works. We simulated five-year periods with present-day temperatures and with a warmer climate, and we investigated how the UP-simulated clouds responded to climate – the ‘cloud feedback’ problem. We found little response of clouds at all latitudes to the imposed climate change, which is within the range of predictions of conventional global climate models. UP is a variation on superparameterization, in which small cloud-resolving models (CRMs) are embedded in each column of the global model. In UP, the CRM grid is fine enough (250 m horizontal × 20 m vertical) to explicitly capture boundary-layer turbulent eddies and associated clouds. Because only one small columnar patch is simulated within each climate model grid cell, this is a million-fold more efficient than simulating the entire globe on this same CRM grid., but achieves much of the same effect for the cloud properties. It doesn’t work perfectly. For instance, like conventional climate models, UP simulates too little subtropical stratocumulus cloud, a bias that we are continuing to work to reduce. However, because it directly simulates the turbulent cloud-forming processes, UP is inherently more plausible for simulating how clouds will change in a perturbed climate. In future, we hope to apply UP to another key climate modeling issue: cloud-aerosol interaction and the effect of human-produced aerosols on the climate change we have already experienced and that which is likely to come.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Landmark-embedded Gaussian process with applications for functional data modeling

In practice, we often need to infer the value of a target variable from functional observation data. A challenge in this task is that the relationship between the functional data and the target variable is very complex: the target variable not only influences the shape but also the location of the functional data. In addition, due to the uncertainties in the environment, the relationship is probabilistic, that is, for a given fixed target variable value, we still see variations in the shape and location of the functional data. To address this challenge, we present a landmark-embedded Gaussian process model that describes the relationship between the functional data and the target variable. A unique feature of the model is that landmark information is embedded in the Gaussian process model so that both the shape and location information of the functional data are considered simultaneously in a unified manner. Gibbs-Metropolis-Hasting algorithm is used for model parameters estimation and target variable inference. The performance of the proposed framework is evaluated by extensive numerical studies and a case study of nano-sensor calibration.

42 ENGINEERING↗
DOE OSTI2013

Embedded Sensors and Controls to Improve Component Performance and Reliability - System Dynamics Modeling and Control System Design

This report documents the current status of the modeling, control design, and embedded control research for the magnetic bearing canned rotor pump being used as a demonstration platform for deeply integrating instrumentation and controls (I&C) into nuclear power plant components. This pump is a highly inter-connected thermo/electro/mechanical system that requires an active control system to operate. Magnetic bearings are inherently unstable system and without active, moment by moment control, the rotor would contact fixed surfaces in the pump causing physical damage. This report details the modeling of the pump rotordynamics, fluid forces, electromagnetic properties of the protective cans, active magnetic bearings, power electronics, and interactions between different dynamical models. The system stability of the unforced and controlled rotor are investigated analytically. Additionally, controllers are designed using proportional derivative (PD) control, proportional integral derivative (PID) control, voltage control, and linear quadratic regulator (LQR) control. Finally, a design optimization problem that joins the electrical, mechanical, magnetic, and control system design into one problem to balance the opposing needs of various design criteria using the embedded system approach is presented.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

OptML

OptML is an open-source Python package that provides modeling and solution capabilities for optimization problems with embedded machine learning models. OptML is a collaborative open-source project that includes advanced algorithms from the academic community for efficient solution of these problems. Sandia contributions to this package include the development of interfaces between these algorithms and Pyomo. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-10861 O

Jalving, Jordan↗
DOE OSTI2020

GMRES with embedded ensemble propagation for the efficient solution of parametric linear systems in uncertainty quantification of computational models

In a previous work, embedded ensemble propagation was proposed to improve the efficiency of sampling-based uncertainty quantification methods of computational models on emerging computational architectures. It consists of simultaneously evaluating the model for a subset of samples together, instead of evaluating them individually. A first approach introduced to solve parametric linear systems with ensemble propagation is ensemble reduction. In Krylov methods for example, this reduction consists in coupling the samples together using an inner product that sums the sample contributions. Ensemble reduction has the advantages of being able to use optimized implementations of BLAS functions and having a stopping criterion which involves only one scalar. However, the reduction potentially decreases the rate of convergence due to the gathering of the spectra of the samples. In this paper, we investigate a second approach: ensemble propagation without ensemble reduction in the case of GMRES. This second approach solves each sample simultaneously but independently to improve the convergence compared to ensemble reduction. This raises two new issues which are solved in this paper: the fact that optimized implementations of BLAS functions cannot be used anymore and that ensemble divergence, whereby individual samples within an ensemble must follow different code execution paths, can occur. We tackle those issues by implementing a high-performing ensemble GEMV and by using masks. The proposed ensemble GEMV leads to a similar cost per GMRES iteration for both approaches, i.e. with and without reduction. For illustration, we study the performances of the new linear solver in the context of a mesh tying problem. Furthermore, this example demonstrates improved ensemble propagation speed-up without reduction.

BLAS↗
DOE OSTI2019

Cluster embedding of ionic systems: Point charges and extended ions

The embedding of cluster models of oxides with point charges and with extensions of the embedding which take into account the spatial extent of the cations is examined with an emphasis on the consequences of this embedding for the relative ionization and excitation energies that are measured in core-level spectroscopies. It is found that the dependence of the electronic structure of the oxides and the relative energies of different levels depend only weakly on the embedding and that relatively simple embeddings may be sufficient to provide an adequate model for determining core-level spectra. This is different from absolute values of the ionizations which, as expected, depend strongly on the details of the extended crystal; however, relative values of binding energies, as measured in photoemission, are of greater interest than the absolute values.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRSDate not supplied

ES2Vec: Earth Science Metadata Suggestions and Analogical Reasoning

As the volume of text-based Earth science research grows, it is increasingly possible to discover latent relationships in the literature. However, traditional methodologies are restricted by limited computational capabilities and intractable problem spaces. Advancements in natural language processing (NLP) have allowed us to use an extensive Earth science corpus to create a domain-specific word vector model, Es2Vec, which we have used to surface latent relationships between Earth science concepts and generate improved keyword tags. Earth science metadata keyword assignment is a challenging problem. Dataset curators select appropriate keywords from the Global Change Master Directory (GCMD) set of keywords. The keywords an are integral part of the search and discovery of these datasets. Hence, the selection of keywords is crucial to increasing the discoverability of datasets. Utilizing machine learning techniques, we provide users with automated keyword suggestions to complement manual selection. We trained a machine learning model that leverages the semantic embedding ability of Word2Vec models to process abstracts and suggest relevant keywords. A user interface tool we built to assist data curators in the assignment of such keywords is also described.

word vectors↗
DOE OSTI2014

Interface scattering in polycrystalline thermoelectrics

We study the effect of electron and phonon interface scattering on the thermoelectric properties of disordered, polycrystalline materials (with grain sizes larger than electron and phonons' mean free path). Interface scattering of electrons is treated with a Landauer approach, while that of phonons is treated with the diffuse mismatch model. The interface scattering is embedded within a diffusive model of bulk transport, and we show that, for randomly arranged interfaces, the overall system is well described by effective medium theory. Using bulk parameters similar to those of PbTe and a square barrier potential for the interface electron scattering, we identify the interface scattering parameters for which the figure of merit ZT is increased. We find the electronic scattering is generally detrimental due to a reduction in electrical conductivity; however, for sufficiently weak electronic interface scattering, ZT is enhanced due to phonon interface scattering.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2024

Accurate assessment of land–atmosphere coupling in climate models requires high-frequency data output

Land–atmosphere (L–A) interactions are important for understanding convective processes, climate feedbacks, the development and perpetuation of droughts, heatwaves, pluvials, and other land-centered climate anomalies. Local L–A coupling (LoCo) metrics capture relevant L–A processes, highlighting the impact of soil and vegetation states on surface flux partitioning and the impact of surface fluxes on boundary layer (BL) growth and development and the entrainment of air above the BL. A primary goal of the Climate Process Team in the Coupling Land and Atmospheric Subgrid Parameterizations (CLASP) project is parameterizing and characterizing the impact of subgrid heterogeneity in global and regional Earth system models (ESMs) to improve the connection between land and atmospheric states and processes. A critical step in achieving that aim is the incorporation of L–A metrics, especially LoCo metrics, into climate model diagnostic process streams. However, because land–atmosphere interactions span timescales of minutes (e.g., turbulent fluxes), hours (e.g., BL growth and decay), days (e.g., soil moisture memory), and seasons (e.g., variability in behavioral regimes between soil moisture and latent heat flux), with multiple processes of interest happening in different geographic regions at different times of year, there is not a single metric that captures all the modes, means, and methods of interaction between the land and the atmosphere. And while monthly means of most of the LoCo-relevant variables are routinely saved from ESM simulations, data storage constraints typically preclude routine archival of the hourly data that would enable the calculation of all LoCo metrics. Here, we outline a reasonable data request that would allow for adequate characterization of sub-daily coupling processes between the land and the atmosphere, preserving enough sub-daily output to describe, analyze, and better understand L–A coupling in modern climate models. A secondary request involves embedding calculations within the models to determine mean properties in and above the BL to further improve characterization of model behavior. Higher-frequency model output will (i) allow for more direct comparison with observational field campaigns on process-relevant timescales, (ii) enable demonstration of inter-model spread in L–A coupling processes, and (iii) aid in targeted identification of sources of deficiencies and opportunities for improvement of the models.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2024

Accurate Assessment of Land-Atmosphere Coupling in Climate Models Requires High Frequency Data Output

Land-atmosphere (L-A) interactions are important for understanding convective processes, climate feedbacks, the development and perpetuation of droughts, heatwaves, pluvials, and other land-centred climate anomalies. Local L-A coupling (LoCo) metrics capture relevant L-A processes, highlighting the impact of soil and vegetation states on surface flux partitioning, and the impact of surface fluxes on boundary layer (BL) growth, development, and entrainment of air above the BL. A primary goal of the Climate Process Team on Coupling Land and Atmospheric Subgrid Parameterizations (CLASP) is parameterizing and characterizing the impact of subgrid heterogeneity in global and regional earth system models (ESMs) to improve the connection between land and atmospheric states and processes. A critical step in achieving that aim is the incorporation of L-A metrics, especially LoCo metrics, into climate model diagnostic process streams. However, because land-atmosphere interactions span time scales of minutes (e.g., turbulent fluxes), hours (e.g., BL growth and decay), days (e.g., soil moisture memory), and seasons (e.g., variability of behavioural regimes between soil moisture and latent heat flux), with multiple processes of interest happening in different geographic regions at different times of year, there is not a single metric that captures all the modes, means, and methods of interaction between the land and the atmosphere. And while monthly means of most of the LoCo-relevant variables are routinely saved from ESM simulations, data storage constraints typically preclude routine archival of the hourly data that would enable the calculation of all LoCo metrics. Here we outline a reasonable data request that would allow for adequate characterization of sub-daily coupling processes between the land and the atmosphere, preserving enough sub-daily output to describe, analyse, and better understand L-A coupling in modern climate models. A secondary request involves embedding calculations within the models to determine mean properties in and above the BL to further improve characterization of model behaviour. Higher-frequency model output will (i) allow for more direct comparison with observational field campaigns on process-relevant time scales, (ii) enable demonstration of inter-model spread in L-A coupling processes, and (iii) aid in targeted identification of sources of deficiencies and opportunities for improvement of the models.

Kirsten L. Findell↗
DOE OSTI2023

Gray-Box Modeling for Distribution Systems with Inverter-Based Resources

In this paper, we develop a novel gray-box modeling approach for distribution systems with inverter-based resources (IBRs). The proposed gray-box modeling method aims to improve estimation accuracy by taking advantages of both physics-based (white-box) and data-driven (black-box) modeling approaches. To this end, the gray-box modeling framework is constructed by encoding prior physical knowledge of the system into a whitebox model and then embedding the output variables of the white-box model into the input vector of a black-box model. Especially, the white-box modeling component is constructed alongside the equivalent network model simplified with Kron reduction. Furthermore, case studies demonstrate that our graybox modeling approach effectively improves estimation accuracy compared to purely physics-based or data-driven methods.

Zhang, Junhui↗
DOE OSTI2015

Flipped GUT inflation

We analyse the prospects for constructing hybrid models of inflation that provide a dynamical realisation of the apparent closeness between the supersymmetric GUT scale and the possible scale of cosmological inflation. In the first place, we consider models based on the flipped SU(5)×U(1) gauge group, which has no magnetic monopoles. In one model, the inflaton is identified with a sneutrino field, and in the other model it is a gauge singlet. In both cases we find regions of the model parameter spaces that are compatible with the experimental magnitudes of the scalar perturbations, A{sub s}, and the tilt in the scalar perturbation spectrum, n{sub s}, as well as with an indicative upper limit on the tensor-to-scalar perturbation ratio, r. We also discuss embeddings of these models into SO(10), which is broken at a higher scale so that its monopoles are inflated away.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1991

A model for the evaporation of clusters of drops embedded in jet vortices. I - Steady injection of identical clusters

A model is developed for describing the interaction of vortex-drop clusters in a flowing gaseous jet, convecting downstream from an injection location. Results are presented for a stationary case representing the situation when identical clusters are continuously injected and the injection rate is constant. The results indicate that, in a rich mixture high-drop-number density regime, the mass evaporated from the drops controls the velocity of the cluster-in-vortex as it propagates downstream.

Bellan, J.↗
NASA NTRS2020

Integrated Modeling and Development of Component-Based Embedded Software in Scala

Programming of embedded systems is challenging due to the low-level design patterns normally applied in the implementation of such systems. Furthermore, programming languages normally considered suitable for this level of programming, such as C and C++, are themselves low-level compared to more modern programming languages. We report on an effort exploring modeling and programming of embedded systems in modern high-level programming languages combining object-oriented and functional programming. We present an integration of four separate internal DSLs (libraries), considered useful for embedded program- ming, in the Scala programming language, for programming and testing component-based systems. These include a DSL for defining components and their connections, and a DSL for programming the individual components as hierarchical state machines. Two additional DSLs support testing, and include a DSL for writing temporal logic flavored test oracles for monitoring program executions, and a DSL for rule-based test input generation. The paper discusses the gap between Scala as used here and the needs for embedded systems programming.

Bocchino, Robert↗