Search NASA⌕ Search

SEARCH · Search NASA

Results for “Hybrid Transformer Model System”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Geothermal well testing pressure prediction by using a hybrid transformer model system: FORGE well use case

Geothermal has huge potential to become an indispensable component in achieving the goal of sustainable energy economy, given its capability to provide consistent baseload power to the electric grid. Injection tests are crucial in geothermal energy system as they naturally help to evaluate reservoir properties, understand fluid flow and even enhance reservoir performance. In this research, we developed a hybrid model system that integrates machine learning (ML) regression, a physics-based mathematical model, and transformer deep learning. Trained and validated using FORGE injection test dataset, this system can forecast the pressure variations both upward and downward over time. The pressure prediction achieved prediction accuracy within 3-6% variance of true pressure values. The system can significantly save time and reduce costs by testing only a few cycles and then using model predictions for further analysis, instead of conducting additional real injection cycle tests. The developed model system also holds promise for designing injection test processes and maintaining well production in geothermal energy. Presented at the IMAGE ‘25 Conference led by Shell.

FORGE↗
DOE OSTI2024

ORBIT: Oak Ridge Base Foundation Model for Earth System Predictability

Earth system predictability is challenged by the complexity of environmental dynamics and the multitude of variables involved. Current AI foundation models, although advanced by leveraging large and heterogeneous data, are often constrained by their size and data integration, limiting their effectiveness in addressing the full range of Earth system prediction challenges. To overcome these limitations, we introduce the Oak Ridge Base Foundation Model for Earth System Predictability (ORBIT), an advanced vision transformer model that scales up to 113 billion parameters using a novel hybrid tensor-data orthogonal parallelism technique. As the largest model of its kind, ORBIT surpasses the current climate AI foundation model size by a thousandfold. Performance scaling tests conducted on the Frontier supercomputer have demonstrated that ORBIT achieves 684 petaFLOPS to 1.6 exaFLOPS sustained throughput, with scaling efficiency maintained at 41% to 85% across 49,152 AMD GPUs. These breakthroughs establish new advances in AI-driven climate modeling and demonstrate promise to significantly improve the Earth system predictability.

Wang, Xiao↗
DOE OSTI2026

Hybrid Oscillator-Qubit Quantum Processors: Instruction Set Architectures, Abstract Machine Models, and Applications

This tutorial offers a pedagogical guide to hybrid quantum processors that integrate discrete-variable (DV) qubits and continuous-variable (CV) oscillators. Aimed at computer scientists, engineers, and physicists, it provides an overview of the experimental, algorithmic, and architectural aspects of this novel and rapidly developing hardware model. Experimental realizations of this model include superconducting, trapped-ion, and neutral-atom platforms. By combining DV and CV components, hybrid oscillator-qubit processors enable a powerful new paradigm that offers complementary strengths for quantum control, error correction, computation, and simulation. Working toward the goal of a full-stack system connecting applications to CV-DV hardware, we define and formulate abstract machine models and instruction set architectures. These essential abstractions enable codesign of hardware and software, and resource estimation for exploring the potential of current and future hardware for computational and simulation tasks. Using these abstractions, we present both new and existing examples that illustrate the benefits of hybrid CV-DV processors relative to traditional DV-only hardware in computation as well as quantum simulation of physical models. Examples include algorithms for transferring states between DV and CV systems, performing the quantum Fourier transform, and simulation of lattice gauge theories. Relative to qubit-only hardware, the bosonic degrees of freedom natively available in hybrid architectures can substantially reduce the circuit complexity of simulations for physical models containing bosons. A key technique is the extension of quantum signal processing ideas to CV-DV systems. This work is intended to serve as a timely and comprehensive guide to this relatively unexplored yet promising approach to quantum computation and to provide a road map to guide future development.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Infiltration synthesis of hybrid nanocomposite resists for advanced nanolithography

In this work, we demonstrate a simple ex-situ inorganic infiltration route for transforming standard organic resists into highperformance positive tone hybrid resist platform. A model thin film PMMA-AlOx hybrid resist system has been synthesized by hybridization of PMMA with AlOx and investigated for electron beam lithography. The approach possesses full controllability of the resist performance in terms of critical does, patterning contrast reaching up to 30 and etch resistance for plasma-based pattern transfer processes. The high selectivity Si etching capability demonstrated using a lowtemperature cryo-Si etch process, based on the controlled infiltration outperforms commercial resists and typical hard mask material thermal SiO 2 , with estimated achievable selectivity in excess of ~300. Si nanostructures down to ~30 nm with aspect ratio up to ~17 are also transferred into the Si substrate. Easy implementation and adaptability for different inorganic infiltrations, this platform is well capable of potentially delivering the resist performance and throughput necessary for EUV lithography.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2022

Tri-level hybrid interval-stochastic optimal scheduling for flexible residential loads under GAN-assisted multiple uncertainties

Various building loads, such as heating, ventilation, and air conditioners (HVACs), electric water heaters (EWHs), and electric vehicles (EVs), can introduce opportunities for improving the flexibility of electricity consumption while satisfying the needs of building owners as well as benefiting the resilience of distribution system. To utilize such flexibility, a tri-level distribution market framework is established, including residential consumers, load aggregators (LAs), and the distribution system operator (DSO). In this work, the uncertainties from all three levels are considered. The random consumption behavior at the consumer level is modeled as a Gaussian noise that is also aggregated and transmitted to the LA level. The weather temperature in the LA level is forecasted as an interval, and the photovoltaic (PV) power in the market-clearing level is modeled by a set of power scenarios generated by Generative Adversarial Networks (GANs). Then, a hybrid interval-stochastic programming is proposed to transform the uncertain problems in the first two levels into deterministic ones. For real-time implementations, a rolling horizon optimization (RHO) scheme is employed to continuously optimize the power consumption based on the latest operating information. Finally, case studies on a modified IEEE 69-bus system validate the effectiveness of the proposed uncertainty modeling strategies and the RHO scheme.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

EV Forecasting-Based Model Predictive Control for Distribution System Congestion Mitigation

The uncoordinated charging of electric vehicles (EVs) in time and space brings congestion issues to the distribution network. This paper proposes an EV charging demand forecasting-based model predictive control (MPC) method for distribution system congestion management. To effectively forecast the time-series EV station charging demand, a hybrid forecasting model that integrates the long short-term memory network (LSTM) and Transformer is proposed. The Transformer-LSTM model is trained using a one-year real historical charging dataset of EV stations to forecast future charging demand in 15-minute intervals. This informs the MPC for distribution network congestion management and minimization of PV curtailment. Numerical results carried out on the modified IEEE 123-bus distribution system demonstrate that the proposed method can effectively resolve line congestion issues through EV smart charging and PV curtailment while outperforming other benchmarks.

ADVANCED PROPULSION SYSTEMS,SOLAR ENERGY↗
DOE OSTI2022

Vulnerability and resilience of urban energy ecosystems to extreme climate events: A systematic review and perspectives

We reviewed the present studies on the vulnerability and resilience of the energy ecosystem (most parts of the energy ecosystem), considering extreme climate events. This study revealed that the increased interactions formed during the transformation of the energy landscape into an ecosystem could notably increase the vulnerability of the energy infrastructure. Such complex ecosystem cannot be assessed using the present state of the art models used by the energy system modelers. Therefore, this study introduces a novel analogy known as the COVID analogy to understand the propagation of disruption within and beyond the energy ecosystem and organized the present state of the art based on the COVID analogy. The analogy helps to categorize the vulnerability of the energy infrastructure into three stages. The study revealed that although there are many publications covering the vulnerability and resilience of the energy infrastructure, considering extreme climate events, the majority are focused on the direct impact of extreme climate on the energy ecosystem. In addition, most of the studies do not consider the impact of future climate variations during this assessment. The propagation of disruptions was assessed mainly for wildfires and hurricanes. Further, there is a clear research gap in considering vulnerability assessment for interconnected energy infrastructure. Here, the transformation of energy systems into a complex ecosystem notably increases the complexity, making it difficult to assess vulnerability and resilience. A shift from a centralized to decentralized modeling architecture could be beneficial when considering the complexities brought by that transformation. Hybrid models consisting of both physical and data-driven machine learning techniques could also be beneficial in this context.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Renewable Thermal Energy Systems: Characterization of the Most Important Thermal Energy Applications in Buildings and Industry (Report 1)

This report is the first in a three-report series that evaluates the provision of renewable heat for industry and buildings via current and prospective renewable thermal energy system (RTES) technologies. The RTES project has undertaken initial research focused on technologies that could be suited for industrial process heat applications at different temperature levels, and, where possible, gathered performance and cost data for these technologies. This project does not directly evaluate RTES for distributed residential or commercial applications, nor does it yet include documented cases or modeling of RTES using geothermal, biomass, waste heat, renewable fuels like renewable natural gas, or hydrogen production. The three technical reports are summarized as follows: Renewable Thermal Energy Systems: Characterization of the Most Important Thermal Energy Applications in Buildings and Industry (Report 1), this report: summary of thermal demands of U.S. industry and buildings, and relevant hybrid RTES configurations; Renewable Thermal Energy Systems: Systemic Challenges and Transformational Policies (Report 2): discussion of socio-technical characteristics of RTES, innovation challenges, and supporting policies. Available at: https://www.nrel.gov/docs/fy23osti/83020.pdf; Renewable Thermal Energy Systems: Modeling Developments and Future Directions (Report 3): Energy yield and performance modeling of RTES, techno-economic analysis via case studies, and proposed development of a user decision support tool. Available at: https://www.nrel.gov/docs/fy23osti/83021.pdf.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2019

Combining anaerobic digestion and hydrothermal liquefaction in the conversion of dairy waste into energy: A techno economic model for New York state

There is increasing global interest and policies being enacted to lower greenhouse gas emissions, especially from the agricultural sector. In the U.S. for example, states with large dairy operations may combine proven manure valorization technologies, such as anaerobic digestion and hydrothermal liquefaction. Sustainable manure treatment would increase the recovery of energy and other useful co-products, namely biogas, biocrude oil and hydro-char as well as lower the environmental impacts. In this study, the economic feasibility of implementing a centralized bioenergy system in New York State was investigated. Here, the feasibility of this transformation depends on many factors, including capital costs, discount rates, and other financing arrangements, electricity selling prices, incentives and farm sizes and locations. For a large-scale implementation in New York State accounting for nearly 50% of the state’s dairy farms, our model of a distributed, hybrid anaerobic/hydrothermal system was shown to treat 590 million liters of wet manure per day, producing 607 million kWh of electricity, 162,000 L of biocrude oil and 117,000 kg of hydro-char per day. Electricity selling price is a critical factor. Increasing the electricity selling price from wholesale ($\$0.06$ /kWh) to retail ($\$0.18$ kWh) increased the net present value from $\$395$ million to $1.5 billion (considering a 40-year project lifetime).

42 ENGINEERING↗
DOE OSTI2022

User guide for the Geospatial Risk Analysis Tool (V.1.0.1)

Developed by the Pacific Northwest National Laboratory (PNNL), the Geospatial Risk Analysis Tool (GRAT) links to Hybrid Single-Particle Lagrangian Integrated Trajectory (HYSPLIT), a complete system for computing simple air parcel trajectories, as well as complex transport, dispersion, chemical transformation, and deposition simulations. The HYSPLIT model was developed by the National Oceanic and Atmospheric Administration (NOAA) Air Resources Laboratory and the Australian Bureau of Meteorology Research Center in 1998. GRAT is used to batch process meteorological data into ARL format. While HYSPLIT GUI limits users to convert meteorological data up to 6 date time points per operation, GRAT enables users to convert a significantly large amount of data (For example, 10 years of meteorological data measured with an interval of 5 minutes) within one operation. Incorporated with the Potential source distribution function (PSDF), GRAT can also be used to determine the areas influenced by the emission of hazardous chemicals.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Variational Adiabatic Gauge Transformation on Real Quantum Hardware for Effective Low-Energy Hamiltonians and Accurate Diagonalization

Effective low-energy theories represent powerful theoretical tools to reduce the complexity in modeling interacting quantum many-particle systems. However, common theoretical methods rely on perturbation theory, which limits their applicability to weak interactions. Here we introduce the Variational Adiabatic Gauge Transformation (VAGT), a nonperturbative hybrid quantum algorithm that can use nowadays quantum computers to learn the variational parameters of the unitary circuit that brings the Hamiltonian to either its block-diagonal or full-diagonal form. If a Hamiltonian can be diagonalized via a shallow quantum circuit, then VAGT can learn the optimal parameters using a polynomial number of runs. The accuracy of VAGT is tested through numerical simulations, as well as simulations on Rigetti and IonQ quantum computers.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

EUTERPE: A global gyrokinetic code for stellarator geometry

The current state of the EUTERPE code is described with emphasis on the implemented models and their numerical implementation. The code solves the multi-species electromagnetic gyrokinetic equations in the full volume of a three-dimensional domain. Noise reduction of the particle-in-cell method is achieved by using a δf-method and Fourier filters. The field equations are discretized with B-splines and the resulting system of equations is solved iteratively. For linear simulations a phase-factor transformation is applied in order to strongly reduce the necessary grid resolution. Apart from the full gyrokinetic model, other numerically less expensive hybrid models are also implemented. They are mainly tailored for comparison with fluid theory and for studying the interaction of the bulk plasma with fast particles. The code is parallelized for CPUs by particle and domain decomposition. Good scalability up to several thousand nodes is demonstrated.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A Digital Twin Framework Utilizing Machine Learning for Robust Predictive Maintenance: Enhancing Tire Health Monitoring

We introduce a novel digital twin (DT) framework for the predictive maintenance of long-term physical systems. Using monitoring tire health as an application, we show how the DT framework can be used to enhance automotive safety and efficiency, and how the technical challenges can be overcome using a three-step approach. First, to manage the data complexity over a long operation span, we employ data reduction techniques to concisely represent physical tires using historical performance and usage data. Relying on these data, for fast real-time prediction, we train a transformer-based model offline on our concise dataset to predict future tire health over time, represented as remaining casing potential (RCP). Based on our architecture, our model quantifies both epistemic and aleatoric uncertainties, providing reliable confidence intervals around predicted RCP. Second, to incorporate real-time data, we update the predictive model in the DT framework, ensuring its accuracy throughout its lifespan with the aid of hybrid modeling and the use of the discrepancy function. Third, to assist decision-making in predictive maintenance, we implement a tire state decision algorithm, which strategically determines the optimal timing for tire replacement based on RCP forecasted by our transformer model. This approach ensures that our DT accurately predicts system health, continually refines its digital representation, and supports predictive maintenance decisions. Furthermore, our framework effectively embodies a physical system, leveraging big data and machine learning (ML) for predictive maintenance, model updates, and decision-making.

advanced computing infrastructure↗
DOE OSTI2022

Solid-State Transformer and Hybrid Transformer With Integrated Energy Storage in Active Distribution Grids: Technical and Economic Comparison, Dispatch, and Control

Solid-state transformer (SST) and hybrid transformer (HT) are promising alternatives to the line-frequency transformer (LFT) in smart grids. The SST features medium-frequency isolation, full controllability for voltage regulation, reactive power compensation, and the capability of battery energy storage system (BESS) integration with multiport configuration. The HT has a partially-rated converter for fractional controllability and can integrate a small BESS. Fast grid-edge voltage fluctuations from increased solar photovoltaic (PV) and electric vehicle (EV) penetration are difficult to manage for mechanical load tap changers. Hence, along with the trend towards more BESS in the grid, the controllability and the storage integration capability of the SST and HT are of strong interest. However, a review of literature shows existing SST and HT research is mostly at converter level, while system-level assessments are scarce. Assessing technical and economic impacts is critical to understanding the benefits and role of the SST and HT to guide future research, which is presented for the first time in this article. Experimental results from medium-voltage (MV) SST and MV HT prototypes are shown to confirm equipment-level feasibility, where the voltage controllability waveforms of a MV HT prototype are reported for the first time. Comparative simulations are performed on a modified IEEE 34-bus system. Here, a grid-model-less decentralized grid-edge voltage control method and a day-ahead BESS dispatch method are proposed for the SST and HT. The simulations show that the SST and HT with integrated storage can host more PV, achieve peak shaving, mitigate voltage fluctuation and reverse power flow, and support energy arbitrage for operational cost reduction, as compared to the LFT. Moreover, comprehensive analyses of net present value (NPV) and internal rate of return (IRR) are performed under different installed PV capacities, HT’s partial converter ratings, and BESS capacities. Sensitivities to future cost reductions of the PV and BESS are studied. Although the NPV and IRR are currently negative, 60% capital cost reduction or 150% revenue increase will make the SST and HT economically viable in the use case studied.

14 SOLAR ENERGY↗
DOE OSTI2023

eYGFPuv-Assisted Transgenic Selection in Populus deltoides WV94 and Multiplex Genome Editing in Protoplasts of P. trichocarpa × P. deltoides Clone ‘52-225’

Although CRISPR/Cas-based genome editing has been widely used for plant genetic engineering, its application in the genetic improvement of trees has been limited, partly because of challenges in Agrobacterium-mediated transformation. As an important model for poplar genomics and biotechnology research, eastern cottonwood (Populus deltoides) clone WV94 can be transformed by A. tumefaciens, but several challenges remain unresolved, including the relatively low transformation efficiency and the relatively high rate of false positives from antibiotic-based selection of transgenic events. Moreover, the efficacy of CRISPR-Cas system has not been explored in P. deltoides yet. Here, we first optimized the protocol for Agrobacterium-mediated stable transformation in P. deltoides WV94 and applied a UV-visible reporter called eYGFPuv in transformation. Our results showed that the transgenic events in the early stage of transformation could be easily recognized and counted in a non-invasive manner to narrow down the number of regenerated shoots for further molecular characterization (at the DNA or mRNA level) using PCR. We found that approximately 8.7% of explants regenerated transgenic shoots with green fluorescence within two months. Next, we examined the efficacy of multiplex CRISPR-based genome editing in the protoplasts derived from P. deltoides WV94 and hybrid poplar clone ‘52-225’ (P. trichocarpa × P. deltoides clone ‘52-225’). The two constructs expressing the Trex2-Cas9 system resulted in mutation efficiency ranging from 31% to 57% in hybrid poplar clone 52-225, but no editing events were observed in P. deltoides WV94 transient assay. The eYGFPuv-assisted plant transformation and genome editing approach demonstrated in this study has great potential for accelerating the genome editing-based breeding process in poplar and other non-model plants species and point to the need for additional CRISPR work in P. deltoides.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Electromagnetic coupling in tight-binding models for strongly correlated light and matter

In this work, we discuss the construction of low-energy tight-binding Hamiltonians for condensed-matter systems with a strong coupling to the quantum electromagnetic field. Such Hamiltonians can be obtained by projecting the continuum theory on a given set of Wannier orbitals. However, different representations of the continuum theory lead to different low-energy formulations because different representations may entangle light and matter, transforming orbitals into light-matter hybrid states before the projection. In particular, a multicenter Power-Zienau-Woolley transformation yields a dipolar Hamiltonian which incorporates the light-matter coupling via both Peierls phases and a polarization density. We compare this dipolar gauge Hamiltonian and the straightforward Coulomb gauge Hamiltonian for a one-dimensional solid to describe subcycle light-driven electronic motion in the semiclassical limit and a coupling of the solid to a quantized cavity mode which renormalizes the band-structure into electron-polariton bands. Both descriptions yield the same result when many bands are taken into account but the dipolar Hamiltonian is more accurate when the model is restricted to few electronic bands, while the Coulomb Hamiltonian requires fewer electromagnetic modes.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2026

Transformer-based operator learning framework for self-energy in strongly correlated systems

We introduce Σ-Attention, a transformer-based operator-learning framework for approximating the self-energy operator of strongly correlated electronic systems. By creating a batched dataset that combines results from three complementary approaches, i.e., many-body perturbation theory, strong-coupling expansion, and exact diagonalization, each effective in specific parameter regimes, Σ-Attention is applied to learn an accurate approximation for the self-energy operator that is valid across a wide range of parameter regimes. This hybrid strategy leverages the strengths of existing methods while relying on the transformer's ability to generalize beyond individual limitations. More importantly, the scalability of the transformer architecture allows the learned self-energy to be extended to systems with larger sizes, leading to much improved computational scaling. Using the one-dimensional Hubbard model, we demonstrate that Σ-Attention can accurately predict the Matsubara Green's function of large systems with a wide range of coupling strength. Our framework offers a promising and scalable pathway for studying strongly correlated systems with many possible generalizations.

Zhu, Yuanran↗
DOE OSTI2022

Imaging Wavepackets in Real-Space & Disentangling Ultrafast Solvation Dynamics with High-Energy Ultrafast X-ray Scattering

The microscopic information of solute and solute-solvent motions can be measured using ultrafast diffuse x-ray scattering, and compared to molecular dynamics simulations, as demonstrated in studies of solvated molecular systems done in X-ray free-electron lasers. However, for the typical photon energies used in these experiments, the photon momentum transfer range was limited to lower Q values where the signals arising from different parts of the studied system overlap significantly. Recently, this limitation was greatly alleviated by extending the photon energy at the Linac Coherent Light-Source (LCLS), significantly increasing the accessible momentum transfer range. This improvement is transformative for ultrafast diffuse x-ray scattering measurements, enabling for the first time to disentangle the solute scattering difference signal, that persists to higher Q values, from the bulk solvent and solute-solvent cross-terms difference signals. In addition, it enhances the prediction ability of modeling and simulation methods such as the hybrid QM/MM approach. The extended Q range also opens the way to resolve in real-space details regarding coherent wavepackets motions beyond their center-of-mass positions. Here, we present the first results on high-energy (18keV) diffuse x-ray scattering in solution, demonstrating high fidelity time-resolved scattering and analysis of the photoexcited model photocatalysts PtPOP ([Pt2(POP)4]4-), and IrDimen ([Ir2(dimen)4]2+) in several solvents. These complexes provide ideal systems for demonstrating the ability of high-Q ultrafast scattering and QM/MM simulations to decompose ultrafast solvation dynamics into specific changes in the solute-solvent pair distribution function with a particular focus on how electronically excited states change the interaction between the solvent and photo-catalytically active metal sites. We introduce an approach to further utilize the high-energy capability and demonstrate a single-shot ultra-wide-angle X-ray Scattering modality using two perpendicular detectors, spanning a scattering angle range of more than 100 degrees, allowing to extend the accessible momentum transfer range up to Q~14 Å-1. We develop a model-free method to invert the x-ray scattering signals and enable the recovery of multiple pair-density motions that happen simultaneously, allowing us to directly measure in real-space nuclear wavepacket motions. . [1] Natan, Adi. "Real-Space Inversion and Super-Resolution of Ultrafast X-ray Scattering using Natural Scattering Kernels." arXiv preprint arXiv:2107.05576 (2021)

Natan, Adi↗