Search NASA⌕ Search

SEARCH · Search NASA

Results for “transformer encoder”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Orbital design of flat bands in non-line-graph lattices via line-graph wave functions

Line-graph (LG) lattices are known for having flat bands (FBs) from the destructive interference of Bloch wave functions encoded in only lattice symmetry. In this work, we develop a generic atomic/molecular orbital design principle for FBs in non-LG lattices. Based on linear combination of atomic orbital theory, we demonstrate that the underlying wave-function symmetry of FBs in a LG lattice can be transformed into the atomic/molecular orbital symmetry in a non-LG lattice. We illustrate such orbital-designed topological FBs in three 2D non-LG, square, trigonal, and hexagonal lattices, where the designed orbitals faithfully reproduce the corresponding lattice symmetries of checkerboard, kagome, and diatomic-kagome lattices, respectively. Interestingly, systematic design of FBs with a high Chern number is also achieved based on the same principle. Fundamentally our theory enriches the FB physics; practically, it significantly expands the scope of FB materials, since most materials have multiple atomic/molecular orbitals at each lattice site, rather than a single s orbital mandated in graph theory and generic lattice models.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2026

Challenges and Vision for Standardization of Biopolymer Data Sets for Machine Learning

Machine learning (ML) is transforming materials research, yet potential for biopolymer discovery remains constrained by fragmented data and nonstandardized reporting. Biopolymers differ significantly from synthetic polymers, requiring specialized approaches to represent their biosynthetic origins, hierarchical structures, and application-specific metrics. In this Perspective, we identify three core challenges limiting biopolymer representation: information encoding, data quality, and data sharing. We describe the most pressing issues and propose commensurate approaches to address each key challenge. Recommendations include the design and adoption of biopolymer-specific fingerprinting and representation frameworks, development of hybrid human-large language model (LLM) data extraction strategies, and expanding Findable, Accessible, Interoperable, Reusable (FAIR)-compliant repositories. We propose a robust foundation to define interoperable, high-quality data sets that capture the full context of biopolymer materials. Standardized metadata, shared ontologies, and community-driven infrastructure would enable scalable, reproducible workflows and accelerate the ML-driven development of biopolymers.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Geospatial Data Platform for All

Spatiotemporal data has evolved in scale due to augmented use in cross-domain applications. Simultaneously, there is substantial growth in the availability of Geographic Information Systems (GIS) data provided by the United States Geological Survey (USGS) along with other federal, state, county, or local agencies through open-data portals and public access APIs. However, data availability does not equate with accessibility. Large-scale analyses and applications require robust, performant data management with co-location of data storage and computing. The insufficiency of data management infrastructure compels researchers to adopt ad hoc project- specific GIS data storage solutions (e.g., copying data to High-Performance computer file systems). As an ad hoc storage strategy does not scale, it hampers cross-domain analyses causing difficulty in data reuse and utilizing existing code bases. Furthermore, GIS data is complex and requires expertise to analyze and manipulate due to its intricate data structures and data-specific projection transformations. Despite the challenges, we recognize that derived GIS data products, e.g., satellite or LIDAR-based images, can be used in downstream applications such as AI by domain, but non-GIS experts. To address the data needs and overcome the challenges, we are working towards a GIS Data Platform focused on efficient data storage, data discovery and access, and an API to enable common workflows. We propose a knowledge-graph (KG) approach for data discovery, whereby datasets are semantically linked to higher- level constructs such as projects and research areas. The semantic data links enable researchers to explore datasets in a top-down approach by specifying relevant and meaningful terms (assists in finding hidden data). An advantage is that the nodes and edges in a knowledge graph create built-in semantic documentation. Deeper spatiotemporal connections between data sources can be encoded via Graph Neural Networks (GNN) (Zhang et al., 2021). The KG approach can be extended to integrate the data itself in a Virtual KG (VKG). Our work will derive inspiration from large-scale VKG efforts that have been undertaken or are currently underway as part of the OpenStreetMap project (Ding et al., 2021). For DOE Data Days, we share the proposed geospatial data platform hybrid (cloud/on-prem) architecture, our work-to-date on storing, retrieving, and transforming LiDAR and raster data relevant to two important NREL use-cases, including the Renewable Energy Potential (reV) Model, and present our proposal for a KG based data discovery engine.

data platform↗
DOE OSTI2022

The Arabidopsis T‐DNA mutant SALK_008491 carries a 14‐kb deletion on chromosome 3 that provides rare insights into the plant response to dynamic light stress

Abstract In nature, plants experience rapid changes in light intensity and quality throughout the day. To maximize growth, they have established molecular mechanisms to optimize photosynthetic output while protecting components of the light‐dependent reaction and CO 2 fixation pathways. Plant phenotyping of mutant collections has become a powerful tool to unveil the genetic loci involved in environmental acclimation. Here, we describe the phenotyping of the transfer‐DNA (T‐DNA) insertion mutant line SALK_008491, previously known as nhd1‐1 . Growth in a fluctuating light regime caused a loss in growth rate accompanied by a spike in photosystem (PS) II damage and increased non‐photochemical quenching (NPQ). Interestingly, an independent nhd1 null allele did not recapitulate the NPQ phenotype. Through bulk sequencing of a backcrossed segregating F 2 pool, we identified an ~14‐kb large deletion on chromosome 3 (Chr3) in SALK_008491 affecting five genes upstream of NHD1 . Besides NHD1 , which encodes for a putative plastid Na + /H + antiporter, the stromal NAD‐dependent D‐3‐phosphoglycerate dehydrogenase 3 ( PGDH3 ) locus was eradicated. Although some changes in the SALK_008491 mutant's photosynthesis can be assigned to the loss of PGDH3, our follow‐up studies employing respective single mutants and complementation with overlapping transformation‐competent artificial chromosome (TAC) vectors reveal that the exacerbated fluctuating light sensitivity in SALK_008491 mutants result from the simultaneous loss of PGDH3 and NHD1. Altogether, the data obtained from this large deletion‐carrying mutant provide new and unintuitive insights into the molecular mechanisms that function to protect the photosynthetic machinery. Moreover, our study renews calls for caution when setting up reverse genetic studies using T‐DNA lines. Although second‐site insertions, indels, and SNPs have been reported before, large deletion surrounding the insertion site causes yet another problem. Nevertheless, as shown through this research, such unpredictable genetic events following T‐DNA mutagenesis can provide unintuitive insights that allow for understanding complex phenomena such as the plant acclimation to dynamic high light stress.

Lopez, Laura S.↗
DOE OSTI2025

Population-level control of two manganese oxidases expands the niche for bacterial manganese biomineralization

Abstract The enzymatic oxidation of aqueous divalent manganese (Mn) is a widespread microbial trait that produces reactive Mn(III, IV) oxide minerals. These biominerals drive carbon, nutrient, and trace metal cycles, thus playing important environmental and ecological roles. However, the regulatory mechanisms and physiological functions of Mn biomineralization are unknown. This challenge arises from the common occurrence of multiple Mn oxidases within the same organism and the use of Mn oxides as indicators of combined gene activity. Through the detection of gene activation in individual cells, we discover that expression ofmnxGandmcoA, two Mn oxidase-encoding genes inPseudomonas putidaGB-1, is confined to subsets of cells within the population, with each gene showing distinct spatiotemporal patterns that reflect local microenvironments. These coordinated intra-population dynamics control Mn biomineralization and illuminate the strategies used by microbial communities to dictate the extent, location, and timing of biogeochemical transformations.

Biotechnology & Applied Microbiology↗
DOE OSTI2024

Massive twistor worldline in electromagnetic fields

We study the (ambi-)twistor model for spinning particles interacting via electromagnetic field, as a toy model for studying classical dynamics of gravitating bodies including effects of both spins to all orders. We compute the momentum kick and spin kick up to one-loop order and show precisely how they are encoded in the classical eikonal. The all-orders-in-spin effects are encoded as a dynamical implementation of the Newman-Janis shift, and we find that the expansion in both spins can be resummed to simple expressions in special kinematic configurations, at least up to one-loop order. We confirm that the classical eikonal can be understood as the generator of canonical transformations that map the in-states of a scattering process to the out-states. We also remark that cut contributions for converting worldline propagators from time-symmetric to retarded amount to the iterated action of the leading eikonal at one-loop order.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Quantum simulations of nuclear resonances with variational methods

Background: The many-body nature of nuclear physics problems poses significant computational challenges. These challenges become even more pronounced when studying the resonance states of nuclear systems, which are governed by the non-Hermitian Hamiltonian. Quantum computing, particularly for quantum many-body systems, offers a promising alternative, especially within the constraints of current noisy intermediate-scale quantum (NISQ) devices. Purpose: This work aims to simulate nuclear resonances using quantum algorithms by developing a variational framework compatible with non-Hermitian Hamiltonians and implementing it fully on a quantum simulator. Methods: We employ the complex scaling technique to extract resonance positions classically and adapt it for quantum simulations using a two-step algorithm. First, we transform the non-Hermitian Hamiltonian into a Hermitian form by using the energy variance as a cost function within a variational framework. Second, we perform 𝜃-trajectory calculations to determine optimal resonance positions in the complex energy plane. To address resource constraints on NISQ devices, we utilize Gray code (GC) encoding to reduce qubit requirements. Results: We first validate our approach using a schematic potential model that mimics a nuclear potential, successfully reproducing known resonance energies with high fidelity. We then extend the method to a more realistic 𝛼−𝛼 nuclear potential and compute the 𝐷- and 𝐺-wave resonance energies with a basis size of 𝑁=16, using only four qubits. The quantum simulation results closely match the classical values, demonstrating the feasibility of our approach. Conclusions: This study demonstrates, for the first time, that the complete 𝜃-trajectory method can be implemented on a quantum computer without relying on any classical input beyond the Hamiltonian. The results establish a scalable and efficient quantum framework for simulating resonance phenomena in nuclear systems. This work represents a significant step toward quantum simulations of open quantum systems and lays the foundation for future investigations into resonance structures in nuclear, atomic, and molecular physics.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

All Optical Neural Networks for Low Power Edge Computing

We developed a simplistic physics-based model of an all-optical neural network that mimics the encoder part of an autoencoder neural network for image compression. Our approach relies on the generation of a MATLAB-based model for both data compression and decompression and utilizes MATLAB's built-in autoencoder networks in combination with simple propagation of optical fields between layers constituting phase elements via Fourier transform. We optimize the phase elements using the particle swarm optimization technique and using our model, we demonstrate a compression ratio of 25% for 2828-pixel input images containing numeric digits from 0 to 9.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Building Neutron Scattering Infrastructure in Louisiana for Advanced Materials (Final Report)

The main objective of the Louisiana Consortium for Neutron Scattering (LaCNS) is to build a major neutron scattering infrastructure capable of treating both soft and hard materials. The goal is to create a sustainable effort having the quality, breadth, and depth necessary to produce competitive proposals including collaborative and center type proposals. Our scientific aim is to understand the role of coupling in emergent complex materials and its impact on the structure/property relationship and to explore how to tune the key couplings to guide the design of materials with the desired properties. This naturally includes building a base of users of the Spallation Neutron Source (SNS) and the High Flux Isotope Reactor (HFIR) in Louisiana; to train highly talented graduate and post-doctoral students in synthesis, neutron scattering, and simulation and modeling techniques, thereby helping to produce the next generation of scientist who use neutron scattering techniques as a crucial part of their research. The goal of our hard matter program is to understand the interplay between spin, charge, orbital, and lattice degrees of freedom in carefully selected complex materials. We have made considerable progress on number of complex systems including the oxide Sr3(Ru1-xMnx)2O7 where short-range magnetic ordering with anisotropic spin texture is initiated at the metal-insulator transition that clearly indicates a strong spin-charge coupling. Another critical area is in quantum materials. For example, in the topological semimetal BaMnSb2, we found a 3D canted antiferromagnetic Weyl semimetal with a 2D electronic structure and a nontrivial Berry phase. In addition, in the hexagonal chiral Mn1/3NbS2 system, we found soliton/soliton and soliton/antisoliton domain walls where the application of small fields or small currents can be used to control nanoscopic magnetic domains where the control of domain walls is crucially important for information storage. In addition, chemical transformation investigations were performed on the VISION instrument at Spallation Neutron Source (SNS). The soft matter effort was focused on understanding the role of non-covalent interactions on the structure and dynamics of fluid-based soft matter. One key focus was on sequence-defined (SD) amphiphilic peptoid polymers that allow encoding of molecular interactions and thereby systematic investigations of how charge directs the solution self-assembly of amphiphilic polymers in water. Another important area was on the dynamics of lipids self-assembled into membranes for exploring the permeability and mechanical using both neutron spin echo (NSE) spectroscopy and quasielastic neutron scattering (QENS) to distinguished between viscoelasticity and permeability at the molecular scale. Overall, the LaCNS project was quite successful, generating 145 publications and 245 presentations. Our graduate and post-doctoral students were well trained resulting in positions in national laboratories (Oak Ridge National Laboratory (ORNL), Argonne National Laboratory and Los Alamos National Laboratory), major research universities, and industry. We also developed a uniaxial pressure cell along with ORNL for SNS. A critical goal of this project was to establish a foundation for competing nationally in federally funded research programs. To this end, we were quite successful in generating over twenty-two federally and non-federally funded grants including awards from NSF, DOE, and DOD, and two early career awards. Equally important, we were able to secure a key major piece of instrumentation via a large ARO grant for a state-of-the-art electron microscope.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Solving high-dimensional inverse problems using amortized likelihood-free inference with noisy and incomplete data

Here, we present a likelihood-free probabilistic inversion method based on normalizing flows for high-dimensional inverse problems. The proposed method is composed of two complementary networks: a summary network for data compression and an inference network for parameter estimation. The summary network encodes raw observations into a fixed-size vector of summary features, while the inference network generates samples of the approximate posterior distribution of the model parameters based on these summary features. The posterior samples are produced in a deep generative fashion by sampling from a latent Gaussian distribution and passing these samples through an invertible transformation. We construct this invertible transformation by sequentially alternating conditional invertible neural network and conditional neural spline flow layers. The summary and inference networks are trained simultaneously. We apply the proposed method to an inversion problem in groundwater hydrology to estimate the posterior distribution of the log-conductivity field conditioned on spatially sparse time-series observations of the system’s hydraulic head responses. The conductivity field is represented with 706 degrees of freedom in the considered problem. Comparison with the likelihood-based iterative ensemble smoother PEST-IES method demonstrates that the proposed method accurately estimates the parameter posterior distribution and the observations’ predictive posterior distribution at a fraction of the inference time of PEST-IES.

conditional invertible neural network↗
DOE OSTI2024

Measurement of the small-scale 3D Lyman- α forest power spectrum

Small-scale correlations measured in the Lyman-α (Lyα) forest encode information about the intergalactic medium and the primordial matter power spectrum. In this article, we present and implement a simple method to measure the 3-dimensional power spectrum, P 3D , of the Lyα forest at wavenumbers k corresponding to small, ~ Mpc scales. In order to estimate P 3D from sparsely and unevenly distributed data samples, we rely on averaging 1-dimensional Fourier Transforms, as previously carried out to estimate the 1-dimensional power spectrum of the Lyα forest, P 1D . Further, this methodology exhibits a very low computational cost. We confirm the validity of this approach through its application to Nyx cosmological hydrodynamical simulations. Subsequently, we apply our method to the eBOSS DR16 Lyα forest sample, providing as a proof of principle, a first P 3D measurement averaged over two redshift bins z = 2.2 and z = 2.4. This work highlights the potential for forthcoming P 3D measurements, from upcoming large spectroscopic surveys, to untangle degeneracies in the cosmological interpretation of P 1D .

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

PDF Entity Annotation Tool (PEAT)

While different text mining approaches – including the use of Artificial Intelligence (AI) and other machine based methods - continue to expand at a rapid pace, the tools used by researchers to create the labeled datasets required for training, modeling, and evaluation remain rudimentary. Labeled datasets contain the target attributes the machine is going to learn; for example, training an algorithm to delineate between images of a car or truck would generally require a set of images with a quantitative description of the underlying features of each vehicle type. Development of labeled textual data that can be used to build natural language machine learning models for scientific literature is not currently integrated into existing manual workflows used by domain experts. Published literature is rich with important information, such as different types of embedded text, plots, and tables that can all be used as inputs to train ML/natural language processing (NLP) models, when extracted and prepared in machine readable formats. Currently, both normalized data extraction of use to domain experts and extraction to support development of ML/NLP models are labor intensive and cumbersome manual processes. Automatic extraction of data and information from formats such as PDFs that are optimized for layout and human readability, not machine readability. The PDF (Portable Document Format) Entity Annotation Tool (PEAT) was developed with the goal of allowing users to annotate publications within their current print format, while also allowing those annotations to be captured in a machine-readable format. One of the main issues with traditional annotation tools is that they require transforming the PDF into plain text to facilitate the annotation process. While doing so lessens the technical challenges of annotating data, the user loses all structure and provenance that was inherent in the underlying PDF. Also, textual data extraction from PDFs can be an error prone process. Challenges include identifying sequential blocks of text and a multitude of document formats (multiple columns, font encodings, etc.). As a result of these challenges, using existing tools for development of NLP/ML models directly from PDFs is difficult because the generated outputs are not interoperable. We created a system that allows annotations to be completed on the original PDF document structure, with no plain text extraction. The result is an application that allows for easier and more accurate annotations. In addition, by including a feature that grants the user the ability to easily create a schema, we have developed a system that can be used to annotate text for different domain-centric schemas of relevance to subject matter experts. Different knowledge domains require distinct schemas and annotation tags to support machine learning.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Engineering crassulacean acid metabolism (CAM) pathways in plants

Disclosed herein are method of altering CAM pathways in plants. In some examples, a disclosed method includes overexpressing one or more genes encoding one or more enzymes that carry out the basic biochemical sequence of nocturnal CO2 fixation (carboxylation) into C4 acids (malate), store C4 acids in the vacuole of the plant, and/or then decarboxylate and refix the released CO2 by C3 photosynthesis during the subsequent day in a plant cell, thereby altering CAM in the plant cell. Also disclosed herein are isolated polynucleotide sequences, transformation vectors, transgenic plant cells, plant part, and plants. The disclosed methods and compositions can be used to improve the water-use efficiency and drought tolerance and durability of plants, such as in plants in arid environments, and also enhance the ability of plants to perform.

Cushman, John C.↗
DOE OSTI2025

Electron-Proton Scattering Event Generation using Structured Tokenization

Recent work such as Omnijet-$\alpha$ has demonstrated that effective tokenization combined with transformer-based architectures can produce effective foundation models for jet physics. While tokenization may help models capture generalizable event characteristics, it also introduces discretization errors that may compromise the precision required for downstream physics analyses. As the number and complexity of the particle features grow, these errors are likely to grow proportionally. In this study, we investigate new tokenization strategies to improve the application of generative transformer models to \textsc{Pythia8} simulations of electron-proton scattering at the Electron-Ion Collider. Specifically, we propose a feature-based structured tokenization approach that utilizes multiple tokens per particle, improving expressivity, while reducing the total number of unique tokens needed. We evaluate this method against grid-based binning, K-means clustering, and vector-quantized variational auto-encoders on the event simulations. Our results show that feature-based structured tokenization reduces discretization error, leading to more accurate generative modeling of particle-level events.

Goldenberg, Steven [Thomas Jefferson National Acce↗
DOE OSTI2025

Electron-Proton Scattering Event Generation using Structured Tokenization

Recent work such as Omnijet-$\alpha$ has demonstrated that effective tokenization combined with transformer-based architectures can produce effective foundation models for jet physics. While tokenization may help models capture generalizable event characteristics, it also introduces discretization errors that may compromise the precision required for downstream physics analyses. As the number and complexity of the particle features grow, these errors are likely to grow proportionally. In this study, we investigate new tokenization strategies to improve the application of generative transformer models to \textsc{Pythia8} simulations of electron-proton scattering at the Electron-Ion Collider. Specifically, we propose a feature-based structured tokenization approach that utilizes multiple tokens per particle, improving expressivity, while reducing the total number of unique tokens needed. We evaluate this method against grid-based binning, K-means clustering, and vector-quantized variational auto-encoders on the event simulations. Our results show that feature-based structured tokenization reduces discretization error, leading to more accurate generative modeling of particle-level events.

Goldenberg, Steven [Thomas Jefferson National Acce↗
DOE OSTI2023

Quantum frequency processor for provable cybersecurity

Methods of quantum key distribution include receiving a frequency bin photon at a location, selecting a frequency bin photon quantum key distribution measurement basis, with a quantum frequency processor, performing a measurement basis transformation on the received frequency bin photon so that the frequency bin photon is measurable in the selected frequency bin photon quantum key distribution measurement basis, and detecting the frequency bin photon in the selected quantum key distribution measurement basis and assigning a quantum key distribution key value based on the detection to a portion of a quantum key distribution key. Apparatus and methods for encoding, decoding, transmitting, and receiving frequency bin photons are disclosed.

Lukens, Joseph M.↗
DOE OSTI2024

Quantum frequency processor for provable cybersecurity

Methods of quantum key distribution include receiving a frequency bin photon at a location, selecting a frequency bin photon quantum key distribution measurement basis, with a quantum frequency processor, performing a measurement basis transformation on the received frequency bin photon so that the frequency bin photon is measurable in the selected frequency bin photon quantum key distribution measurement basis, and detecting the frequency bin photon in the selected quantum key distribution measurement basis and assigning a quantum key distribution key value based on the detection to a portion of a quantum key distribution key. Apparatus and methods for encoding, decoding, transmitting, and receiving frequency bin photons are disclosed.

Lukens, Joseph M.↗
DOE OSTI2022

Efficient hybrid explicit-implicit learning for multiscale problems

Splitting method is a powerful method to handle application problems by splitting physics, scales, domain, and so on. Many splitting algorithms have been designed for efficient temporal discretization. Here, in this paper, our goal is to use temporal splitting concepts in designing machine learning algorithms and, at the same time, help splitting algorithms by incorporating data and speeding them up. We propose a machine learning assisted splitting scheme which improves the efficiency of the scheme meanwhile preserves the accuracy. We consider a recently introduced multiscale splitting algorithms, where the multiscale problem is solved on a coarse grid. To approximate the dynamics, only a few degrees of freedom are solved implicitly, while others explicitly. This splitting concept allows identifying degrees of freedom that need implicit treatment. In this paper, we use this splitting concept in machine learning and propose several strategies. First, the implicit part of the solution can be learned as it is more difficult to solve, while the explicit part can be computed. This provides a speed-up and data incorporation for splitting approaches. Secondly, one can design a hybrid neural network architecture because handling explicit parts requires much fewer communications among neurons and can be done efficiently. Thirdly, one can solve the coarse grid component via PDEs or other approximation methods and construct simpler neural networks for the explicit part of the solutions. We discuss these options and implement one of them by interpreting it as a machine translation task. This interpretation of the splitting scheme successfully enables us using the Transformer since it can perform model reduction for multiple time series and learn the connection between them. We also find that the splitting scheme is a great platform to predict the coarse solution with insufficient information of the target model: the target problem is partially given and we need to solve it through a known problem which approximates the target. Our machine learning model can incorporate and encode the given information from two different problems and then solve the target problems. We conduct four numerical examples and the results show that our method is stable and accurate.

97 MATHEMATICS AND COMPUTING↗