Search NASA⌕ Search

SEARCH · Search NASA

Results for “transformer encoder”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Patch Hierarchical Attention Transformer for Efficient Particle Jet Tagging

Real-time jet tagging is critical for identifying short-lived particle decays in the high-throughput detectors of the Large Hadron Collider, where real-time trigger systems responsible for deciding which collision events to store impose strict latency and accuracy constraints. While transformer architectures achieve the highest jet tagging accuracy when compute is unconstrained, their quadratic self-attention cost makes inference restrictive on trigger budget. Existing efficient variants reduce the computational cost, but hinder the classification performance. To address this limitation, we introduce the Patch Hierarchical Attention Transformer (PHAT-JeT), which combines two mechanisms: a physics-inspired geometric message-passing module that encodes local detector-plane structure, and a hierarchical patch-based attention scheme that computes exact attention within small particle groups while preserving global context through lightweight patch-token communication. Within a restricted budget, PHAT-JeT achieves state-of-the-art accuracy and background rejection among all resource-constrained jet tagging models on four benchmarks (\textsc{hls4ml}, JetClass, Top Tagging, and Quark--Gluon). Our code is available at https://github.com/aaronw5/PHAT-JeT.

Wang, Aaron [Illinois U., Chicago] (ORCID:00000003↗
DOE OSTI2024

Nonlinear encoding in diffractive information processing using linear optical materials

Nonlinear encoding of optical information can be achieved using various forms of data representation. Here, we analyze the performances of different nonlinear information encoding strategies that can be employed in diffractive optical processors based on linear materials and shed light on their utility and performance gaps compared to the state-of-the-art digital deep neural networks. For a comprehensive evaluation, we used different datasets to compare the statistical inference performance of simpler-to-implement nonlinear encoding strategies that involve, e.g., phase encoding, against data repetition-based nonlinear encoding strategies. We show that data repetition within a diffractive volume (e.g., through an optical cavity or cascaded introduction of the input data) causes the loss of the universal linear transformation capability of a diffractive optical processor. Therefore, data repetition-based diffractive blocks cannot provide optical analogs to fully connected or convolutional layers commonly employed in digital neural networks. However, they can still be effectively trained for specific inference tasks and achieve enhanced accuracy, benefiting from the nonlinear encoding of the input information. Our results also reveal that phase encoding of input information without data repetition provides a simpler nonlinear encoding strategy with comparable statistical inference accuracy to data repetition-based diffractive processors. Our analyses and conclusions would be of broad interest to explore the push-pull relationship between linear material-based diffractive optical systems and nonlinear encoding strategies in visual information processors.

42 ENGINEERING↗
DOE OSTI2025

N -way parametric frequency beamsplitter for quantum photonics

Optical networks are the leading platform for the transfer of information due to their low loss and ability to scale to many information channels using optical frequency modes. To fully leverage the quantum properties of light in this platform, it is desired to manipulate higher-dimensional superpositions by orchestrating linear, beamsplitter-type interactions between several channels simultaneously. We propose a method of achieving simultaneous, all-to-all coupling between N optical frequency modes via N -way Bragg-scattering four-wave mixing. By exploiting the frequency degree of freedom, additional modes can be multiplexed in an interaction medium of fixed volume and loss while avoiding the introduction of excess noise. We generalize the theory of the frequency-encoded two-mode interaction to N modes under this four-wave-mixing approach, finding that arbitrary unitary transformations are possible through successive four-wave-mixing pump configurations. We experimentally verify the quantum nature of this scheme by demonstrating three-way multiphoton interference. Two input photons are shared among three frequency modes and display interference differing from that of two classical (coherent-state) inputs. These results show the potential of our approach for the scalability of photonic quantum information processing to general N -mode systems in the frequency domain. Published by the American Physical Society 2025

Oliver, Richard (ORCID:0000000207760657)↗
DOE OSTI2026

Emerging Flexible Designs for Geospatial Multimodal Foundation Models

Foundation models are rapidly transforming Earth observation by enabling scalable pretraining across diverse unlabeled geospatial modalities. However, their architectural diversity—ranging from encoder-only to encoder-decoder and masked autoencoding paradigms—makes it challenging to assess performance trade-offs in a consistent manner. In this work, we present an apples-to-apples comparison of leading FM architectures designed for geospatial multimodal reasoning, with a particular focus on flexibility across varied spectral band configurations. We standardize pretraining using identical self-supervised learning objectives and training datasets, and evaluate all models under consistent parameterization on the GEOBench benchmark across classification and segmentation tasks. Our results offer new insights into the design trade-offs between model flexibility, modality alignment, and downstream task performance. By highlighting architectural strengths and limitations under controlled conditions, this study provides practical guidance for building next-generation geospatial foundation models capable of robust multimodal reasoning.

Ambrozio Dias, Philipe [ORNL] (ORCID:0000000194277↗
DOE OSTI2026

Quantitative Dissection of Agrobacterium Virulence to Generate a Synthetic Ti Plasmid

Agrobacterium is not only a costly plant pathogen but is also an essential tool for plant transformation. Though Agrobacterium-mediated transformation (AMT) has been heavily studied, its polygenic nature and complex transcriptional regulation make identification of the genetic basis of transformational efficiency difficult through traditional genetic and bioinformatic approaches. Here, we use a bottom-up synthetic approach to systematically engineer the tumor-inducing plasmid (pTi), wherein the majority of virulence machinery is encoded. Using a validated toolkit to control Agrobacterium gene expression in planta, we perform a quantitative dissection of AMT to investigate the contributions of critical vir-genes at different expression levels. We construct a synthetic pTi capable of transient plant and stable fungal transformation and characterize bottlenecks and solutions for complex polygenic synthetic pTi designs. Our reductionist approach demonstrates how bottom-up engineering can be used to dissect and elucidate the genetic underpinnings of complex biological traits, laying the foundation for future engineering to establish full synthetic control over the critical process of AMT.

Agrobacterium-mediated transformation↗
DOE OSTI2025

SigTime: Learning and Visually Explaining Time Series Signatures

Understanding and distinguishing temporal patterns in time series data is essential for scientific discovery and decision-making. For example, in biomedical research, uncovering meaningful patterns in physiological signals can improve diagnosis, risk assessment, and patient outcomes. However, existing methods for time series pattern discovery face major challenges, including high computational complexity, limited interpretability, and difficulty in capturing meaningful temporal structures. Here, to address these gaps, we introduce a novel learning framework that jointly trains two Transformer models using complementary time series representations: shapelet-based representations to capture localized temporal structures and traditional feature engineering to encode statistical properties. The learned shapelets serve as interpretable signatures that differentiate time series across classification labels. Additionally, we develop a visual analytics system—SigTime—with coordinated views to facilitate exploration of time series signatures from multiple perspectives, aiding in useful insights generation. We quantitatively evaluate our learning framework on eight publicly available datasets and one proprietary clinical dataset. Additionally, we demonstrate the effectiveness of our system through two usage scenarios along with the domain experts: one involving public ECG data and the other focused on preterm labor analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Automating NISQ Application Design with Meta Quantum Circuits with Constraints (MQCC)

Near-term intermediate scale quantum (NISQ) computers are likely to have very restricted hardware resources, where precisely controllable qubits are expensive, error-prone, and scarce. Programmers of such computers must therefore balance trade-offs among a large number of (potentially heterogeneous) factors specific to the targeted application and quantum hardware. To assist them, we propose Meta Quantum Circuits with Constraints (MQCC), a meta-programming framework for quantum programs. Programmers express their application as a succinct collection of normal quantum circuits stitched together by a set of (manually or automatically) added meta-level choice variables, whose values are constrained according to a programmable set of quantitative optimization criteria. MQCC’s compiler generates the appropriate constraints and solves them via an SMT solver, producing an optimized, runnable program. We showcase a few MQCC’s applications for its generality including an automatic generation of efficient error syndrome extraction schemes for fault-tolerant quantum error correction with heterogeneous qubits and an approach to writing approximate quantum Fourier transformation and quantum phase estimation that smoothly trades off accuracy and resource use. We also illustrate that MQCC can easily encode prior one-off NISQ application designs-–multi-programming (MP), crosstalk mitigation (CM)—as well as a combination of their optimization goals (i.e., a combined MP-CM).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Strategies for simulating the time evolution of Hamiltonian lattice field theories

Simulating the time evolution of quantum field theories given some Hamiltonian H requires developing algorithms for implementing the unitary operator e -iHt . A variety of techniques exist that accomplish this task, with the most common technique used so far being Trotterization, which is a special case of the application of a product formula. However, other techniques exist that promise better asymptotic scaling in certain parameters of the theory being simulated, the most efficient of which are based on the concept of block encoding. In this work we study the performance of such algorithms in simulating lattice field theories. We derive and compare the asymptotic gate complexities of several commonly used simulation techniques in application to Hamiltonian lattice field theories. Using the scalar $\hat{φ}$ 4 theory as a test, we also perform numerical studies and compare the gate costs required by product formulas and signal-processing-based techniques to simulate time evolution. For the latter, we use the linear combination of unitaries (LCU) construction augmented with the quantum Fourier transform circuit to switch between the field and momentum eigenbases, which leads to immediate order-of-magnitude improvement in the cost of preparing the block encoding. Further, this paper also includes a pedagogical review of the techniques used, in particular product formulas, LCU, qubitization, quantum signal processing, as well as the technique for simulating geometrically-local Hamiltonians developed by Haah, Hastings, Kothari, and Low.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

A fast Fourier transform-based solver for elastic micropolar composites

This work presents a spectral micromechanical formulation for obtaining the full-field and homogenized response of elastic micropolar composites. The algorithm relies on a coupled set of convolution integral equations for the micropolar strains, where periodic Green’s operators associated with a linear homogeneous reference medium are convolved with functions of the Cauchy and couple stress fields that encode the material’s heterogeneity, as well as any potential material nonlinearity. Such convolution integral equations take an algebraic form in the reciprocal Fourier space that can be solved iteratively. In this vein, the fast Fourier transform (FFT) algorithm is leveraged to accelerate the numerical solution, resulting in a mesh-free formulation in which the periodic unit cell representing the heterogeneous material can be discretized by a regular grid of pixels in two dimensions (or voxels in three dimensions). For verification, the numerical solutions obtained with the micropolar FFT solver are compared with analytical solutions for a matrix with a dilute circular inclusion subjected to plane strain loading. The developed computational framework is then used to study length-scale effects and effective (micropolar) moduli of composites with various topological configurations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Equivalence between Fermion-to-Qubit Mappings in two Spatial Dimensions

We argue that all locality-preserving mappings between fermionic observables and Pauli matrices on a two-dimensional lattice can be generated from the exact bosonization in Chen et al., whose gauge constraints project onto the subspace of the toric code with emergent fermions. Starting from the exact bosonization and applying Clifford finite-depth generalized local unitary transformation, we can achieve all possible fermion-to-qubit mappings (up to the re-pairing of Majorana fermions). In particular, we discover a new supercompact encoding using 1.25 qubits per fermion on the square lattice. We prove the existence of finite-depth quantum circuits to obtain fermion-to-qubit mappings with qubit-fermion ratios r = 1 + 1/2k for positive integers k, utilizing the trivialness of quantum cellular automata in two spatial dimensions. Also, we provide direct constructions of fermion-to-qubit mappings with ratios arbitrarily close to 1. When the ratio reaches 1, the fermion-to-qubit mapping reduces to the one-dimensional Jordan-Wigner transformation along a certain path in the two-dimensional lattice. Finally, we explicitly demonstrate that the Bravyi-Kitaev superfast simulation, the Verstraete-Cirac auxiliary method, Kitaev’s exactly solved model, the Majorana loop stabilizer codes, and the compact fermion-to-qubit mapping can all be obtained from the exact bosonization.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Block encoding bosons by signal processing

Block Encoding (BE) is a crucial subroutine in many modern quantum algorithms, including those with near-optimal scaling for simulating quantum many-body systems, which often rely on Quantum Signal Processing (QSP). Currently, the primary methods for constructing BEs are the Linear Combination of Unitaries (LCU) and the sparse oracle approach. In this work, we demonstrate that QSP-based techniques, such as Quantum Singular Value Transformation (QSVT) and Quantum Eigenvalue Transformation for Unitary Matrices (QETU), can themselves be efficiently utilized for BE implementation. Specifically, we present several examples of using QSVT and QETU algorithms, along with their combinations, to block encode Hamiltonians for lattice bosons, an essential ingredient in simulations of high-energy physics. We also introduce a straightforward approach to BE based on the exact implementation of Linear Operators Via Exponentiation and LCU (LOVE-LCU). We find that, while using QSVT for BE results in the best asymptotic gate count scaling with the number of qubits per site, LOVE-LCU outperforms all other methods for operators acting on up to qubits, highlighting the importance of concrete circuit constructions over mere comparisons of asymptotic scalings. Using LOVE-LCU to implement the BE, we simulate the time evolution of single-site and two-site systems in the lattice theory using the Generalized QSP algorithm and compare the gate counts to those required for Trotter simulation.

Kane, Christopher F↗
DOE OSTI2020

Efficient Fourier transforms for transverse momentum dependent distributions

Hadron production at low transverse momenta in semi-inclusive deep inelastic scattering can be described by transverse momentum dependent (TMD) factorization. This formalism has also been widely used to study the Drell-Yan process and back-to-back hadron pair production in $e^+e^-$ collisions. These processes are the main ones for extractions of TMD parton distribution functions and TMD fragmentation functions, which encode important information about nucleon structure and hadronization. One of the most widely used TMD factorization formalism in phenomenology formulates TMD observables in coordinate $b_\perp$-space, the conjugate space of the transverse momentum. The Fourier transform from $b_\perp$-space back into transverse momentum space is sufficiently complicated due to oscillatory integrands that it requires a careful and computationally intensive numerical treatment in order to avoid potentially large numerical errors. Within the TMD formalism, the azimuthal angular dependence is analytically integrated and the two-dimensional $b_\perp$ integration reduces to a one-dimensional integration over the magnitude $b_\perp$. In this paper we develop a fast numerical Hankel transform algorithm for such a $b_\perp$-integration that improves the numerical accuracy of TMD calculations in all standard processes. Libraries for this algorithm are implemented in Python 2.7 and 3, C++, as well as FORTRAN77. All packages are made available open source.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

The anaerobic fungus Neocallimastix californiae shifts metabolism and produces melanin in response to lignin-derived aromatic compounds

Biological deconstruction of lignocellulose for sustainable chemical production offers an opportunity to harness evolutionarily specialized enzymes and organisms for industrial bioprocessing. While hydrolysis of cellulose and hemicellulose by CAZymes yields fermentable sugars, ligninolysis releases a heterogeneous mix of aromatic compounds that likely play a crucial role in shaping microbial communities and microbial metabolism. Here, we interrogated the metabolomic and transcriptomic response of a lignocellulolytic anaerobic fungus, Neocallimastix californiae, to a heterogeneous mixture of aromatic compounds derived from lignin. Through exposing the fungus to both a concentration it might experience in its native environment and an elevated concentration of alkaline lignin, we observe that N. californiae transforms vanillin and that supplying alkaline lignin at 0.125 g/L, alongside cellulose, enhances the growth and polysaccharide-degrading activity of N. californiae. Altogether, our results further suggest that vanillin consumption, increased polymer-degrading activity, increased metabolic activity, and transcriptomic remodeling of amino acid synthesis genes all coincide with increased melanin production by fungal cells. These observations challenge previous notions that aromatics from lignocellulose only inhibit the growth and polymer deconstruction capabilities of the biomass-degrading anaerobic fungi (Neocallimastigomycetes). This study demonstrates that anaerobic fungi have a complex relationship with aromatic chemicals derived from lignin and hemicellulose and shift their metabolism in response to the addition of lignocellulose-derived aromatics to their growth medium. Further, as no known pathways for the biochemical transformation of aromatics were detected in these organisms despite observed transcriptome remodeling in the presence of aromatics, we suggest they might encode novel biochemical routes for scavenging amino acid building blocks from aromatic monomers derived from hemicellulose side chains and lignin.

Anaerobic fungi↗
DOE OSTI2025

Gravitational memory and Ward identities in the local detector frame

Gravitational memory, which describes the permanent shift in the strain after the passage of gravitational waves, is directly related to Weinberg’s soft graviton theorems and the Bondi-Metzner-Sachs (BMS) symmetry group of asymptotically flat space-times. In this work, we provide an equivalent description of the phenomenon in local coordinates around gravitational wave detectors, such as transverse-traceless (TT) gauge. We show that gravitational memory is encoded in large residual diffeomorphisms in this gauge, which include time-dependent anisotropic spatial rescalings, and prove their equivalence to BMS transformations when translated to TT gauge. We then derive the associated Ward identities and associated soft theorems, for both scattering amplitudes and equal-time (in-in) correlation functions, and explicitly check their validity for planar gravitational waves. Furthermore, the in-in identities are recognized as the flat-space analog of the well-known inflationary consistency relations.

General relativity↗
DOE OSTI2026

Design and performance of AI agents interfacing with an atomic layer deposition tool

In this work, we introduce the design of an atomic layer deposition (ALD) reactor augmented with an AI interface for autonomous materials synthesis. Our modular design encapsulates the particularities of the hardware behind a Python interface that communicates with the ALD control software via transmission control protocol. This interface is compatible with model context protocol interfaces used in agentic frameworks. We have integrated our tool with a simple AI agent that leverages a large language model to transform user-supplied queries into ALD processes that are then run in our reactor. Our approach uses a JavaScript object notation schema to encode ALD processes. Our experimental results show that the AI interface does not impose a significant overhead to our control software, at least within our fastest 10 ms scale. We also carried out a detailed evaluation of the agent performance using leading models in two classes of tasks: basic instruction and process discovery tasks, where the agent is presented with a target material and needs to identify the correct ALD process compatible with the reactor configuration. Despite the simplicity of our agent design, we observed that most of the advanced models excelled at the instruction tasks. However, only recent models, such as o1, o3, GPT-5, and Claude Opus 4, performed well in process discovery tasks. We also observed significant variability in the response for the hardest challenges. While the results obtained are promising, we identify areas where AI research could improve the performance of agents for ALD.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Multiscale Data-Driven Seismic Full-Waveform Inversion With Field Data Study

Seismic full-waveform inversion (FWI), which uses iterative methods to estimate high-resolution subsurface models from seismograms, is a powerful imaging technique in exploration geophysics. In recent years, the computational cost of FWI has grown exponentially due to the increasing size and resolution of seismic data. Moreover, it is a nonconvex problem and can encounter local minima due to the limited accuracy of the initial velocity models or the absence of low frequencies in the measurements. To overcome these computational issues, we develop a multiscale data-driven FWI method based on fully convolutional networks (FCNs). In preparing the training data, we first develop a real-time style transform method to create a large set of synthetic subsurface velocity models from natural images. We then develop two convolutional neural networks with encoder-decoder structures to reconstruct the low- and high-frequency components of the subsurface velocity models, separately. To validate the performance of our data-driven inversion method and the effectiveness of the synthesized training set, we compare it with conventional physics-based waveform inversion approaches using both synthetic and field data. Finally, these numerical results demonstrate that, once our model is fully trained, it can significantly reduce the computation time and yield more accurate subsurface velocity models in comparison with conventional FWI.

58 GEOSCIENCES↗
DOE OSTI2022

High-dimensional discrete Fourier transform gates with a quantum frequency processor

The discrete Fourier transform (DFT) is of fundamental interest in photonic quantum information, yet the ability to scale it to high dimensions depends heavily on the physical encoding, with practical recipes lacking in emerging platforms such as frequency bins. In this article, we show that d -point frequency-bin DFTs can be realized with a fixed three-component quantum frequency processor (QFP), simply by adding to the electro-optic modulation signals one radio-frequency harmonic per each incremental increase in d . We verify gate fidelity F W > 0.9997 and success probability P W > 0.965 up to d = 10 in numerical simulations, and experimentally implement the solution for d = 3, utilizing measurements with parallel DFTs to quantify entanglement and perform tomography of multiple two-photon frequency-bin states. Our results furnish new opportunities for high-dimensional frequency-bin protocols in quantum communications and networking.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Why Is Attention Sparse In Particle Transformer?

Transformer-based models have achieved state-of-the-art performance in jet tagging at the CERN Large Hadron Collider (LHC), with the Particle Transformer (ParT) representing a leading example of such models. A striking feature of ParT is its sparse, nearly binary, attention structure, raising questions about the origin of this behavior and whether it encodes physically meaningful correlations. In this work, we investigate the source of ParT's sparse attention by comparing models trained on multiple benchmark datasets and examine the relative contributions of the attention term and the physics-inspired interaction matrix before softmax. We find that binary sparsity arises primarily from the attention mechanism itself, with the interaction matrix playing a secondary role. Moreove, we show that ParT is able to identify key jet substructure elements, such as leptons in semileptonic top decays, even without explicit particle identification inputs. These results provide new insight into the interpretability of transformer-based jet taggers and clarify the conditions under which sparse attention patterns emerge in ParT.

Legge, Timothy [UC, San Diego]↗