Search NASA⌕ Search

SEARCH · Search NASA

Results for “transformer encoder”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

zPerf: A Statistical Gray-Box Approach to Performance Modeling and Extrapolation for Scientific Lossy Compression

With the scaling up of simulation-based scientific discovery on high-performance computing systems, the disparity between compute and I/O has increased, forcing domain scientists to save only a small amount of simulation data to persistent storage. This can result in the loss of essential physics fields that are needed for data analysis. While error-bounded lossy compression has made tremendous progress in bridging the gap between compute and I/O, the lack of understanding of compression performance remains a key hurdle to its wide adoption. Here, in this work, we present zPerf, a statistical gray-box performance modeling approach for scientific lossy compression. Our contributions are threefold: 1) We develop zPerf to estimate the performance of lossy compression techniques, based on in-depth understanding and statistical modeling for data features and core compression metrics; 2) We demonstrate the in-detailed implementation of zPerf using two case studies, where we derive the performance modeling for SZ and ZFP, two leading lossy compressors; 3) We evaluate the effectiveness of zPerf on real-world datasets across various domains. Based on the evaluation, we demonstrate the efficacy of the zPerf performance model; 4) We further discuss three case studies where zPerf is applied to extrapolate the compression ratio of SZ and ZFP with alternative encoding schemes as well as ZFP with an alternative transform scheme. Through the case studies, we demonstrate the potential of zPerf for exploring the design space of lossy compression, which has hardly been studied in the literature.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Methods and compositions for enhanced biomass production and increased abiotic stress tolerance

This invention relates to methods for increasing carbon fixation, increasing biomass production and/or increasing abiotic stress tolerance in a plant comprising: introducing into a plant, plant part, and/or plant cell a heterologous polynucleotide encoding a ferredoxin polypeptide and/or a heterologous polynucleotide encoding a biotin ligase polypeptide to produce a stably transformed plant, plant part, and/or plant cell, wherein said heterologous polynucleotides are from a bacterial and/or an archaeal species. Further provided are plants, plant parts and plant cells produced by the methods of the invention.

Grunden, Amy Michele↗
DOE OSTI2021

The widespread IS200/IS605 transposon family encodes diverse programmable RNA-guided endonucleases

Tracing the origin of CRISPR-Cas CRISPR-Cas systems have transformed genome editing and other biotechnologies; however, the broader origins and diversity of RNA-guided nucleases have largely remained unexplored. Altae-Tran et al . show that three distinct transposon-encoded proteins, IscB, IsrB, and TnpB, are naturally occurring, reprogrammable RNA-guided DNA nucleases (see the Perspective by Rousset and Sorek). In addition to identifying diverse guide-encoding mechanisms, the authors elucidate the evolutionary relationship between IsrB, IscB, and CRISPR-Cas9. Overall, these newly characterized systems, called OMEGA (for obligate mobile element–guided activity) systems, are found in all domains of life and may be harnessed for biotechnology development. —DJ

Science & Technology - Other Topics↗
DOE OSTI2026

An efficient explicit implementation of a near-optimal quantum algorithm for simulating linear dissipative differential equations

We propose an efficient block-encoding technique for the implementation of the Linear Combination of Hamiltonian Simulations (LCHS) for simulating dissipative initial-value problems. This algorithm approximates a target nonunitary operator as a weighted sum of Hamiltonian evolutions, thereby emulating a dissipative problem by mixing various time scales. We introduce an efficient encoding of the LCHS into a quantum circuit based on a simple coordinate transformation that turns the dependence on the summation index into a trigonometric function. Classically, this method is equivalent to the use of a highly accurate Fejér-Clenshaw-Curtis quadrature formula. Quantumly, this significantly simplifies block-encoding of a dissipative problem and allows one to perform an exponential number of Hamiltonian simulations by a single Quantum Signal Processing (QSP) circuit. The resulting LCHS circuit has high success probability and the selector scales logarithmically with the number of terms in the LCHS sum and linearly with time. Careful analysis of error convergence proves that this method is more efficient than other LCHS circuits that have recently appeared in the literature. We verify the quantum circuit and its scaling by simulating it on a digital emulator of fault-tolerant quantum computers and, as a test problem, solve the advection-diffusion equation. The proposed algorithm can be used for simulating a wide class of nonunitary initial-value problems including the Liouville equation with added dissipation and linear embeddings of nonlinear systems, such as the Koopman-von Neumann and Carleman embeddings.

Novikau, I [Lawrence Livermore National Laboratory↗
DOE OSTI2025

A hybrid neural architecture: Online attosecond x-ray characterization

The emergence of high-repetition-rate x-ray free-electron lasers (XFELs), such as SLAC’s LCLS-II, serves as our canonical example for autonomous controls that necessitate high-throughput diagnostics paired with streaming computational pipelines capable of single-shot analysis with extremely low latency. We present the deterministic characterization with an integrated parallelizable hybrid resolver architecture, a hybrid machine learning framework designed for fast, accurate analysis of XFEL diagnostics using angular streaking-based sinogram images. This architecture integrates convolutional neural networks and bidirectional long short-term memory models to denoise input, identify x-ray sub-spike features, and extract sub-spike relative delays with sub-30 attosecond temporal resolution. Deployed on low-latency hardware, it achieves over 10 kHz throughput with 168.3 μs inference latency, indicating scalability to 14 kHz with field-programmable gate array integration. By transforming regression tasks into classification problems and leveraging optimized error encoding, we achieve high precision with low-latency performance that is critical for real-time streaming event selection and experimental control feedback signals. This represents a key development in real-time control pipelines for next-generation autonomous science, generally, and high repetition-rate x-ray experiments in particular.

Accelerator Physics (physics.acc-ph)↗
DOE OSTI2025

Lossy Compression: An Online Multi-Stage Technology for High-Fidelity Synchro- Waveform Measurements

Effective real-time monitoring and analysis of distributed grids necessitate the use of synchro-waveform measurements, which capture almost all high-frequency disturbances and transient phenomena. However, due to limitations in high-speed measurements and network bandwidth, it is challenging to transfer all high-fidelity synchro-waveforms losslessly and successfully. To cope with these challenges, a hybrid-based online multi-stage compression algorithm is proposed to significantly improve the compression efficiency for synchro-waveform measurements. Initially, the multiple discrete Wavelet transformation is deployed to deconstruct the waveform components. The delta encoding is further developed to decrease the magnitude. In conjunction with the Lempel-Ziv-Markov chain, the hybrid compression algorithm is implemented to achieve real-time compression for the synchro-waveform measurements. Moreover, an innovative error index that synergizes the time and frequency domain error and correlation is formulated to evaluate the waveform distortion. By integrating compression ratio, suitable parameters can be optimally selected. Finally, the simulation, laboratory experiments, as well as field tests across a spectrum of sampling frequencies and time intervals are conducted to substantiate the efficacy of the proposed method. Here, the outcomes demonstrated that a compression ratio of approximately 15.5 and 17.83 can be reached for 0.5 s and 1 s data under both offline and online scenarios, which equates to a substantial 93.5% to 94.39% reduction in data storage requirements.

High-fidelity synchro-waveform measurements↗
DOE OSTI2021

Distributed Acoustic Sensing Based on Coherent Microwave Photonics Interferometry

A microwave photonics method has been developed for measuring distributed acoustic signals. This method uses microwave-modulated low coherence light as a probe to interrogate distributed in-fiber interferometers, which are used to measure acoustic-induced strain. By sweeping the microwave frequency at a constant rate, the acoustic signals are encoded into the complex microwave spectrum. The microwave spectrum is transformed into the joint time–frequency domain and further processed to obtain the distributed acoustic signals. The method is first evaluated using an intrinsic Fabry Perot interferometer (IFPI). Acoustic signals of frequency up to 15.6 kHz were detected. The method was further demonstrated using an array of in-fiber weak reflectors and an external Michelson interferometer. Two piezoceramic cylinders (PCCs) driven at frequencies of 1700 Hz and 3430 Hz were used as acoustic sources. The experiment results show that the sensing system can locate multiple acoustic sources. The system resolves 20 nε when the spatial resolution is 5 cm. The recovered acoustic signals match the excitation signals in frequency, amplitude, and phase, indicating an excellent potential for distributed acoustic sensing (DAS).

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Beyond interpolation: Physics-inspired gating transformers for extrapolating irradiation conditions to novel nuclear fuels

The qualification of advanced nuclear fuels relies on irradiation experiments in test reactors that emulate commercial conditions. Designing these tests requires accurate prediction of key irradiation quantities, particularly heat generation rate and burnup, yet obtaining them typically involves computationally expensive multi-step simulation workflows. We propose a physics-inspired gating transformer (PIGT) that integrates an inverse-square, distance-based attenuation into the encoder representation to bias attention toward physically relevant spatial relationships while retaining data-driven flexibility. Using MiniFuel irradiation data from the High Flux Isotope Reactor at Oak Ridge National Laboratory, we benchmark against ensemble methods, feedforward and recurrent networks, convolutional models, and standard transformers. While baseline models perform well under interpolation, they exhibit a pronounced generalization gap when evaluated on fuels not included in the training set. The proposed model consistently improves extrapolative accuracy and stability, yielding the strongest performance on unseen fuel configurations. These results indicate that a lightweight physics structure embedded within attention mechanisms can substantially improve robustness, enabling more reliable surrogate predictions to accelerate the design of nuclear fuel irradiation experiments.

Fuel qualification↗
DOE OSTI2024

Discovery of highly potent and ALK2/ALK1 selective kinase inhibitors using DNA-encoded chemistry technology

Activin receptor type 1 (ACVR1; ALK2) and activin receptor like type 1 (ACVRL1; ALK1) are transforming growth factor beta family receptors that integrate extracellular signals of bone morphogenic proteins (BMPs) and activins into Mothers Against Decapentaplegic homolog 1/5 (SMAD1/SMAD5) signaling complexes. Several activating mutations in ALK2 are implicated in fibrodysplasia ossificans progressiva (FOP), diffuse intrinsic pontine gliomas, and ependymomas. The ALK2 R206H mutation is also present in a subset of endometrial tumors, melanomas, non-small lung cancers, and colorectal cancers, and ALK2 expression is elevated in pancreatic cancer. Using DNA-encoded chemistry technology, we screened 3.94 billion unique compounds from our diverse DNA-encoded chemical libraries (DECLs) against the kinase domain of ALK2. Off-DNA synthesis of DECL hits and biochemical validation revealed nanomolar potent ALK2 inhibitors. Further structure-activity relationship studies yielded center for drug discovery (CDD)-2789, a potent [NanoBRET (NB) cell IC50: 0.54 μM] and metabolically stable analog with good pharmacological profile. Crystal structures of ALK2 bound with CDD-2281, CDD-2282, or CDD-2789 show that these inhibitors bind the active site through Van der Waals interactions and solvent-mediated hydrogen bonds. CDD-2789 exhibits high selectivity toward ALK2/ALK1 in KINOMEscan analysis and NB K192 assay. In cell-based studies, ALK2 inhibitors effectively attenuated activin A and BMP-induced Phosphorylated SMAD1/5 activation in fibroblasts from individuals with FOP in a dose-dependent manner. Thus, CDD-2789 is a valuable tool compound for further investigation of the biological functions of ALK2 and ALK1 and the therapeutic potential of specific inhibition of ALK2.

Jimmidi, Ravikumar↗
DOE OSTI2021

Terpene synthases for biofuel production and methods thereof

The present invention relates to terpene synthases capable of degrading precursors into biofuel compounds, such as terpenoid compounds. In one instance, a transformed organism can include such terpene synthases, as well as vectors encoding such synthases. Methods of employing such synthases and organisms are also described herein.

Wu, Weihua↗
DOE OSTI2022

An efficient and broadly applicable method for transient transformation of plants using vertically aligned carbon nanofiber arrays

Transient transformation in plants is a useful process for evaluating gene function. However, there is a scarcity of minimally perturbing methods for gene delivery that can be used on multiple organs, plant species, and non-excised tissues. We pioneered and demonstrated the use of vertically aligned carbon nanofiber (VACNF) arrays to efficiently perform transient transformation of different tissues with DNA constructs in multiple plant species. The VACNFs permeabilize plant tissue transiently to allow molecules into cells without causing a detectable stress response. We successfully delivered DNA into leaves, roots and fruit of five plant species (Arabidopsis, poplar, lettuce, Nicotiana benthamiana , and tomato) and confirmed accumulation of the encoded fluorescent proteins by confocal microscopy. Using this system, it is possible to transiently transform plant cells with both small and large plasmids. The method is successful for species recalcitrant to Agrobacterium -mediated transformation. VACNFs provide simple, reliable means of DNA delivery into a variety of plant organs and species.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Generalized Quantum Signal Processing

Quantum signal processing (QSP) and quantum singular value transformation (QSVT) currently stand as the most efficient techniques for implementing functions of block-encoded matrices, a central task that lies at the heart of most prominent quantum algorithms. However, current QSP approaches face several challenges, such as the restrictions imposed on the family of achievable polynomials and the difficulty of calculating the required phase angles for specific transformations. In this paper, we present a generalized quantum signal processing (GQSP) approach, employing general SU(2) rotations as our signal-processing operators, rather than relying solely on rotations in a single basis. Our approach lifts all practical restrictions on the family of achievable transformations, with the sole remaining condition being that | P | ≤ 1 , a restriction necessary due to the unitary nature of quantum computation. Furthermore, GQSP provides a straightforward recursive formula for determining the rotation angles needed to construct the polynomials in cases where P and Q are known. In cases where only P is known, we provide an efficient optimization algorithm capable of identifying in under a minute of GPU time, a corresponding Q for polynomials of degree on the order of 10 7 . We further illustrate GQSP simplifies QSP-based strategies for Hamiltonian simulation, offer an optimal solution to the ϵ -approximate fractional query problem that requires O ( ( 1 / δ ) + log ( 1 / ϵ ) ) queries to perform where O ( 1 / δ ) is a proved lower bound, and introduces novel approaches for implementing bosonic operators. Moreover, we propose a novel framework for the implementation of normal matrices, demonstrating its applicability through synthesis of diagonal matrices, as well as the development of a new algorithm for convolution through synthesis of circulant matrices using only O ( d log N + log 2 N ) 1 and 2-qubit gates for a filter of lengths d . Published by the American Physical Society 2024

Motlagh, Danial↗
DOE OSTI2023

Oracle-Preserving Latent Flows

A fundamental task in data science is the discovery, description, and identification of any symmetries present in the data. We developed a deep learning methodology for the simultaneous discovery of multiple non-trivial continuous symmetries across an entire labeled dataset. The symmetry transformations and the corresponding generators are modeled with fully connected neural networks trained with a specially constructed loss function, ensuring the desired symmetry properties. The two new elements in this work are the use of a reduced-dimensionality latent space and the generalization to invariant transformations with respect to high-dimensional oracles. The method is demonstrated with several examples on the MNIST digit dataset, where the oracle is provided by the 10-dimensional vector of logits of a trained classifier. We find classes of symmetries that transform each image from the dataset into new synthetic images while conserving the values of the logits. We illustrate these transformations as lines of equal probability (“flows”) in the reduced latent space. These results show that symmetries in the data can be successfully searched for and identified as interpretable non-trivial transformations in the equivalent latent space.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Deformable phrase level attention: A flexible approach for improving AI based medical coding

Objective: Improving the AI-driven automated medical encoding of clinical text plays a vital role in gathering information on the occurrence of diseases to improve population-level health. This work presents a novel attention mechanism designed to enhance text classification models and ensure appropriate classification of medical concepts in unstructured electronic health records. Materials and Methods: We developed a deformable, phrase-level attention mechanism to identify important lexical word-level and contextual phrase-level information from clinical text documents. We evaluated conventional and transformer-based deep learning models that we extended with our attention mechanism on the extraction of critical cancer information (e.g., site, subsite, laterality, histology, behavior) from 629,908 electronic pathology reports and on the automated medical encoding of 52,722 hospital discharge summaries. Results: Transformer-based models with the deformable, phrase-level attention mechanism achieved the best performance on the extraction of critical cancer information from pathology reports. Conventional- and transformer-based models show similar or better performance than their baseline counterparts on the automated medical encoding of clinical documents. Discussion: The addition of phrase-level information allowed models extended with our proposed method to outperform standard word-level attention. Our method showed favorable properties for the real-world application in terms of model robustness and phenotyping. These results indicate that our method is promising for automated data harmonization for common data models. Conclusion: This work proposes a novel deformable, phrase-level attention mechanism that enhances text classification models in the extraction of medical concepts from clinical text documents. We demonstrate strong performances on two clinical text datasets and showcase real-world deployability of our method.

Automated medical encoding↗
DOE OSTI2026

Peak2Patch: High-Fidelity Functional Group Identification through Attention-Based Fusion of Infrared and Mass Spectra

Identifying molecular structure based on spectroscopic readings is a key task in a variety of chemical and biological applications. Common spectroscopy techniques, such as Infrared (IR) Spectroscopy and Mass Spectrometry (MS), provide detailed information on the structure of molecular compounds but nonetheless require expert-level knowledge to decode. Machine learning has emerged as a potential solution for automating structure prediction from chemical spectra; however, current approaches generally focus on single sensor modalities, neglecting to leverage the complementary information contained within differing spectra. In this paper, we introduce Peak2Patch, a novel approach to fusion-enhanced prediction of functional groups from IR and mass spectra. First, we perform a detailed comparison of backbone networks for encoding both sparse mass spectra and dense IR spectra and demonstrate the superior performance of transformer neural networks over current state-of-the-art convolutional neural networks. Second, we evaluate three broad categories of fusion: early (raw feature), middle (deep feature), and late (decision) fusion, demonstrating the potential of a deep feature fusion-based approach. Lastly, we present Peak2Patch, our attention-based fusion scheme, which leverages cross-attention to mix features between encoded tokens of the two modalities. We validate our approach on a publicly available multimodal spectroscopic data set of 790k simulated molecules, demonstrating a large improvement in functional group prediction over both the previous state-of-the-art and our own strong single-modal baselines.

Jacobson, Philip [Sandia National Laboratories (SN↗
DOE OSTI2023

Modified biosensors and biocatalysts and methods of use

Biosensors including a nucleic acid encoding a PcaU protein, a PobR protein, a CatM protein, a PcaR protein, or a TphR protein are provided. In some examples, the biosensors include a promoter regulated by the sensed molecule operably linked to a reporter gene. The biosensors may be included in a vector or in cells including one or more of the biosensors or vectors. Modified chorismate pyruvate lyase (UbiC) and modified paraoxonase (PON1) proteins including one or more amino acid substitutions are provided. Finally, methods of selecting biocatalysts with increased activity including transforming a library of cells expressing a biosensor with one or more nucleic acids encoding one or more mutations in a gene involved in a biosynthesis pathway, determining activity of the reporter protein; and selecting a cell with increased reporter protein activity as expressing a biocatalyst with increased activity are provided.

Jha, Ramesh K.↗
DOE OSTI2024

Physics-constrained machine learning for electrodynamics without gauge ambiguity based on Fourier transformed Maxwell’s equations

We utilize a Fourier transformation-based representation of Maxwell’s equations to develop physics-constrained neural networks for electrodynamics without gauge ambiguity, which we label the Fourier–Helmholtz–Maxwell neural operator method. In this approach, both of Gauss’s laws and Faraday’s law are built in as hard constraints, as well as the longitudinal component of Ampère–Maxwell in Fourier space, assuming the continuity equation. An encoder–decoder network acts as a solution operator for the transverse components of the Fourier transformed vector potential, $\hat{A}_⟂(k,t)$, whose two degrees of freedom are used to predict the electromagnetic fields. This method was tested on two electron beam simulations. Among the models investigated, it was found that a U-Net architecture exhibited the best performance as it trained quicker, was more accurate and generalized better than the other architectures examined. We demonstrate that our approach is useful for solving Maxwell’s equations for the electromagnetic fields generated by intense relativistic charged particle beams and that it generalizes well to unseen test data, while being orders of magnitude quicker than conventional simulations. We show that the model can be re-trained to make highly accurate predictions in as few as 20 epochs on a previously unseen data set.

97 MATHEMATICS AND COMPUTING↗