Search NASA⌕ Search

SEARCH · Search NASA

Results for “transformer models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Geothermal well testing pressure prediction by using a hybrid transformer model system: FORGE well use case

Geothermal has huge potential to become an indispensable component in achieving the goal of sustainable energy economy, given its capability to provide consistent baseload power to the electric grid. Injection tests are crucial in geothermal energy system as they naturally help to evaluate reservoir properties, understand fluid flow and even enhance reservoir performance. In this research, we developed a hybrid model system that integrates machine learning (ML) regression, a physics-based mathematical model, and transformer deep learning. Trained and validated using FORGE injection test dataset, this system can forecast the pressure variations both upward and downward over time. The pressure prediction achieved prediction accuracy within 3-6% variance of true pressure values. The system can significantly save time and reduce costs by testing only a few cycles and then using model predictions for further analysis, instead of conducting additional real injection cycle tests. The developed model system also holds promise for designing injection test processes and maintaining well production in geothermal energy. Presented at the IMAGE ‘25 Conference led by Shell.

FORGE↗
NASA NTRS1989

A numerical analysis of Stefan problems for generalized multi-dimensional phase-change structures using the enthalpy transforming model

An enthalpy transforming scheme is proposed to convert the energy equation into a nonlinear equation with the enthalpy, E, being the single dependent variable. The existing control-volume finite-difference approach is modified so it can be applied to the numerical performance of Stefan problems. The model is tested by applying it to a three-dimensional freezing problem. The numerical results are in agreement with those existing in the literature. The model and its algorithm are further applied to a three-dimensional moving heat source problem showing that the methodology is capable of handling complicated phase-change problems with fixed grids.

Cao, Yiding↗
NASA NTRS2011

Model Transformation for a System of Systems Dependability Safety Case

The presentation reviews the dependability and safety effort of NASA's Independent Verification and Validation Facility. Topics include: safety engineering process, applications to non-space environment, Phase I overview, process creation, sample SRM artifact, Phase I end result, Phase II model transformation, fault management, and applying Phase II to individual projects.

Murphy, Judy↗
DOE OSTI2026

MTL_TX: A Multi-Task Transformer Model for Improved Radiation Time-Series Estimation

Controlling radiation doses at potential radioactive facilities is critical to ensuring the safety of both personnel and the public. At the Thomas Jefferson National Accelerator Facility (JLab), multiple sensors are deployed around the three experimental halls to monitor key parameters, including single-beam current, energy levels, current leakage, and radiation values during accelerator operations. In this study, we developed a Multi-task Transformer model, MTL_TX, to accurately estimate radiation doses at sensor locations based on historical data, with the aim of enhancing safety in accelerator facilities and surrounding public areas. To improve estimation accuracy, we integrated two innovative components into the proposed model: hierarchical feature embedding (HFE) and multi-level decomposition attention (MDA). Additionally, the multi-task learning (MTL) framework effectively leverages correlations among multiple sensors, enabling individual estimations for each sensor. MTL_TX achieved outstanding results on data collected in 2018, with an MSE of 0.1464, an RMSE of 0.2353, and an R 2 score of 0.8584. Furthermore, when trained on 2018 data, MTL_TX exhibited excellent generalization capability to unseen datasets from 2016 to 2019, achieving an MSE of 0.1407, an RMSE of 0.2263, and an R 2 score of 0.8831. These results demonstrate a significant improvement over existing state-of-the-art models.

Transformer↗
DOE OSTI2024

CovTransformer: A transformer model for SARS-CoV-2 lineage frequency forecasting

With hundreds of SARS-CoV-2 lineages circulating in the global population, there is an ongoing need for predicting and forecasting lineage frequencies and thus identifying rapidly expanding lineages. Accurate prediction would allow for more focused experimental efforts to understand pathogenicity of future dominating lineages and characterize the extent of their immune escape. Here, we first show that the inherent noise and biases in lineage frequency data make a commonly-used regression-based approach unreliable. To address this weakness, we constructed a machine learning model for SARS-CoV-2 lineage frequency forecasting, called CovTransformer, based on the transformer architecture. We designed our model to navigate challenges such as a limited amount of data with high levels of noise and bias. We first trained and tested the model using data from the UK and the USA, and then tested the generalization ability of the model to many other countries and US states. Remarkably, the trained model makes accurate predictions two months into the future with high levels of accuracy both globally (in 31 countries with high levels of sequencing effort) and at the US-state level. Our model performed substantially better than a widely used forecasting tool, the multinomial regression model implemented in Nextstrain, demonstrating its utility in SARS-CoV-2 monitoring. Assuming a newly emerged lineage is identified and assigned, our test using retrospective data shows that our model is able to identify the dominating lineages 7 weeks in advance on average before they became dominant. Overall, our work demonstrates that transformer models represent a promising approach for SARS-CoV-2 forecasting and pandemic monitoring.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Novel Deep Learning Transformer Model for Short to Sub‐Seasonal Streamflow Forecast

Accurate short-to-subseasonal streamflow forecasts are becoming crucial for effective water management in an increasingly variable climate. However, streamflow forecast remains challenging over extended lead times, uncertainty in meteorological inputs, and increased frequency and variability in extreme weather and climate events. We implemented a Future Time Series Transformer (FutureTST) model for streamflow forecasting that separately integrates past meteorological and streamflow data while incorporating future weather conditions. FutureTST achieves a mean Nash-Sutcliffe Efficiency (NSE) of 0.82 to 0.67 for 1- to 30-day streamflow forecasts. Incorporating upstream streamflow information improved forecast accuracy by up to 10%. During real-time forecast, FutureTST maintains higher forecast skills of 9.03 for 1-day and 5.74 for 14-day forecasts. In contrast, calibrated process-based hydrological model forecasts become unreliable beyond a 4-day lead time. Our findings demonstrate the potential of FutureTST as a reliable streamflow forecasting tool that offers a valuable addition to operational flood monitoring systems and climate-resilient decision-making.

Ambika, Anukesh Krishnankutty [Oak Ridge National ↗
DOE OSTI2025

Elasto-viscoplastic fast Fourier transform modeling framework for assessing microstructural effects on stress intensity factors characterizing fracture toughness

A large-strain elasto-viscoplastic fast Fourier transform (LS-EVPFFT) model with non-periodic (NP) velocity-based boundary conditions is adapted to simulate the sensitivity of stress intensity factors on microstructure for 304L stainless steel. The material was characterized via electron backscattered diffraction (EBSD) serial-sectioning to obtain a measured 3-D microstructural cell to perform simulations. The NP-LS-EVPFFT model, including the simulation setup and boundary conditions, was verified using a crystal plasticity finite element (CPFE) model. To this end, the generation of meshes of notched specimens was developed, which involved creating Python scripts for mesh “cutting” in Abaqus, and Sculpt scripts in Cubit for meshing of the measured microstructural cell processed with DREAM.3D. The complexity of the mesh preparation highlighted the advantages of the FFT-based model, which circumvents the mesh generation process. Given the efficiency of the FFT-based model, statistical distribution of stress intensity factors in function of crystal orientation at the crack tip, grain structure, and crystallographic texture surrounding the crack tip were predicted. Further, the distributions reveal about 10% variation of stress intensity factors with microstructure with the most significant sensitivity found to be the crystal orientation at the crack tip. The methodology developed in this work is discussed as a practical simulation tool for predicting the sensitivity of stress intensity factors on microstructural variability in metallic materials.

36 MATERIALS SCIENCE↗
NASA NTRS2013

Transfer Function Identification Using Orthogonal Fourier Transform Modeling Functions

A method for transfer function identification, including both model structure determination and parameter estimation, was developed and demonstrated. The approach uses orthogonal modeling functions generated from frequency domain data obtained by Fourier transformation of time series data. The method was applied to simulation data to identify continuous-time transfer function models and unsteady aerodynamic models. Model fit error, estimated model parameters, and the associated uncertainties were used to show the effectiveness of the method for identifying accurate transfer function models from noisy data.

Morelli, Eugene A.↗
DOE OSTI2026

MATEY: multiscale adaptive transformer models for spatiotemporal physical systems

Accurate representation of the multiscale features in spatiotemporal physical systems using vision transformer architectures requires extremely long, computationally prohibitive token sequences. To address this issue, we propose two novel adaptive tokenization schemes that dynamically adjust patch sizes based on local features: one ensures convergent behavior to uniform patch refinement, while the other offers better computational efficiency. Moreover, we present a set of spatiotemporal attention schemes, where the temporal or axial spatial dimensions are decoupled, to evaluate their baseline computational and data efficiencies and to determine whether adaptive tokenization can improve this performance. We assess the performance of the proposed multiscale adaptive model, MATEY, in a sequence of experiments. Compared to a full spatiotemporal attention scheme or a scheme that decouples only the temporal dimension, we find that fully decoupled axial attention is less efficient and expressive, requiring more training time and model parameters to achieve the same accuracy. The experiments on the adaptive tokenization schemes show that, compared to a uniformly refined model, the proposed schemes achieve comparable or improved accuracy at a much lower cost in the tested two-dimensional settings. While the asymptotic analysis suggests the potential for favorable scaling, empirical validation at substantially longer sequence lengths remains to be performed in future work. Finally, we demonstrate in two fine-tuning tasks featuring different physics that models pretrained on PDEBench data outperform the ones trained from scratch, especially in the low data regime with frozen attention.

adaptive tokenization↗
NASA NTRS2016

Multidisciplinary Model Transformation Through Simplified Intermediate Representations

There has long been a challenge of making engineering tools from multiple disciplines interoperate. This problem extends to system modeling practices. This challenge has been confronted with a wide variety of techniques. These techniques include attempting to interface tools together into combined suites, attempting to find underlying commonalities in mathematics, supporting connections through semantic encoding, various graph mappings and transformations, and code wrappers. All of these approaches have strengths and weaknesses. These are measured in multiple areas: relative freedom of action of individual domain engineers in developing their own tools, speed of execution, ease of creation, traceability, fidelity of information transfer, and degree of alignment between the concepts of different domains. This paper presents an approach to this interoperation problem currently being used in the World-Wide Web. The approach is to develop easy-to-parse formats that allow flexibility to both the file author and file interpreter. Many of the formats that are currently deployed sacrifice runtime performance for the ability of third parties to easily understand what to do with the data. XML became popular earlier as a de-facto standard format for many web applications, but is now being replaced by JSON to enhance human readability and provide a simpler data model. This is the basis for work in this paper. Our approach, which provides the key to interoperation, is a simplified “shrapnel” intermediate collection of objects and relationships that is the result of a breakdown of the system model into minimal pieces. It is then reassembled on the destination side, forming a two-step transformation. Previous efforts with single-step transformations have proven too difficult to create efficiently. In contrast, the use of this approach leads to an almost automatic procedure for transformation development. The Europa project is a large engineering project that must coordinate the efforts of many different teams with different specialties. The traditional form of exchanging engineering information has been documentation. The vision of model-based systems engineering is to make this information exchange much more digital. This paper presents the application of our simplified format to connecting two different engineering tools to the system model, with a focus on a dynamic mission simulation encoded in Modelica.

Cole, Bjorn↗
NASA NTRS2020

The “Gearamid” Model: Transforming NASA Langley’s Role in the Aerospace Technology Ecosystem

NASA’s operating environment is evolving: numerous new emerging technologies are converging to create breakthrough solutions, many nontraditional players are partnering to create those solutions, and a diverse array of new public-private funding models are being employed. A team at NASA’s Langley Research Center (LaRC) developed a model to describe the challenges that must be addressed for an emerging breakthrough technology to penetrate the appropriate market sector, and how all the players in the relevant ecosystem can collaborate to accelerate the market infusion process. This model, which the team dubbed the “Gearamid,” originated from a narrower study to determine what LaRC should do to capitalize on and advance autonomous technology as a “game changer” in the civil aerospace domain. The elements in the original version of the Gearamid depict the various challenges that need to be addressed as an autonomous technology proceeds from initial development to market infusion. In addition, the Gearamid indicates the “actors” best suited to address each challenge element. The study team noted that all challenge elements in the Gearamid must be worked concurrently to assure successful infusion of autonomous solutions. The team also found that NASA expertise naturally positions Agency organizations—including LaRC—to lead contributions that address certain challenge elements and to play a supplemental role in other areas where entities external to NASA are actively working and investing to solve challenges. Furthermore, the team concluded that NASA can play a leadership role in coordinating efforts of the diverse entities across the civil aerospace community. After determining the center’s optimal role in the autonomous technology development ecosystem, the study team then extrapolated the Gearamid model for autonomous technologies into a broader, more general model depicting the challenge elements that must be overcome to develop and infuse any emerging technology. LaRC has embraced the Gearamid model and is using it to drive changes that will transform the center and allow it to function optimally in the evolving landscape. Given LaRC’s successes, the study team suggests that other organizations may also be able to use the Gearamid model to inform future planning/strategy efforts.

Jill M Marlowe↗
DOE OSTI2024

Synthesis of Correct Digital Controller Models from Specifications by Model Transformation (21-0320)

The design of high consequence controllers (in weapons systems, autonomy, etc.) that do what they are supposed to do is a significant challenge. Testing simply does not come close to meeting the requirements for assurance. Today circuit designers at Sandia (and elsewhere) typically capture the core behavior of their components using state models in tools such as STATEFLOW. They then check that their models meet certain requirements (e.g. “The system bus must not deadlock” or “both traffic lights at an intersection must not be green at the same time”) using tools called model checkers. If the model checker returns “yes” then the property is guaranteed to be satisfied by the model. However, there are several drawbacks to this industry practice: (1) there is a lot of detail to get right, this is particularly challenging when there are multiple components requiring complex coordination (2) any errors returned by the model checker have to be traced back through the design and fixed, necessitating rework, (3) there are severe scalability problems with this approach, particularly when dealing with concurrency. All this places high demands on the designers who now face not only an accelerated schedule but also controllers of increasing complexity. This report describes a new and fundamentally different approach to the construction of safety-critical digital controllers. Instead of directly constructing a complete model and then trying to verify it, the designer can start with an initial abstract (think “sketch”) model plus the requirements, from which a correct concrete model is automatically synthesized. There is no need for post-hoc verification of required functional properties. Having tool to carry this out will significantly impact the nation’s ability to ensure the safety of high-consequence digital systems. The approach has been implemented in a prototype tool, along with a suite of examples, including ones that reflect actual problems faced by designers. Our approach operates on a variant of Statecharts developed at Sandia called Qspecs. Statecharts are a widely used formalism for developing concurrent reactive systems, supporting scalability through allowing state models containing composite states, which are the serial or parallel composition of substates which can themselves contain statecharts. Statecharts enable an incremental style of development, in which states are progressively refined to incorporate greater detail in an incremental model of software development. Our approach formulates a set of constraints from the structure of the models and the requirements and propagates these constraints to a fixpoint. The solution to the constraints is an inductive invariant along with guards on the transitions. We also show how our approach extends to implementation refinement, decomposition, composition, and elaboration. We currently handle safety requirements written in LTL (Linear Temporal Logic)

42 ENGINEERING↗
NASA NTRS2010

Model Transformation for a System of Systems Dependability Safety Case

Software plays an increasingly larger role in all aspects of NASA's science missions. This has been extended to the identification, management and control of faults which affect safety-critical functions and by default, the overall success of the mission. Traditionally, the analysis of fault identification, management and control are hardware based. Due to the increasing complexity of system, there has been a corresponding increase in the complexity in fault management software. The NASA Independent Validation & Verification (IV&V) program is creating processes and procedures to identify, and incorporate safety-critical software requirements along with corresponding software faults so that potential hazards may be mitigated. This Specific to Generic ... A Case for Reuse paper describes the phases of a dependability and safety study which identifies a new, process to create a foundation for reusable assets. These assets support the identification and management of specific software faults and, their transformation from specific to generic software faults. This approach also has applications to other systems outside of the NASA environment. This paper addresses how a mission specific dependability and safety case is being transformed to a generic dependability and safety case which can be reused for any type of space mission with an emphasis on software fault conditions.

Murphy, Judy↗
DOE OSTI2025

Phase transformation kinetics model for metals

We develop a new model for phase transformation kinetics in metals by generalizing the Levitas–Preston (LP) phase field model of martensite phase transformations (see Levitas and Preston (2002a,b) and Levitas et al. (2003)) to arbitrary pressure. Furthermore, we account for and track: the interface speed of the pressure-driven phase transformation, properties of critical nuclei, as well as nucleation at grain sites and on dislocations and homogeneous nucleation. The volume fraction evolution of each phase is described by employing KJMA (Kolmogorov, 1937; Johnson and Mehl, 1939; Avrami, 1939, 1940, 1941) kinetic theory. We then test our new model for iron under ramp loading conditions and compare our predictions for the α → ϵ iron phase transition to experimental data of Smith et al. (2013). In conclusion, more than one combination of material and model parameters (such as dislocation density and interface speed) led to good agreement of our simulations to the experimental data, thus highlighting the importance of having accurate microstructure data for the sample under consideration.

36 MATERIALS SCIENCE↗
NASA NTRS1994

Image Discrimination Models Predict Object Detection in Natural Backgrounds

Object detection involves looking for one of a large set of object sub-images in a large set of background images. Image discrimination models only predict the probability that an observer will detect a difference between two images. In a recent study based on only six different images, we found that discrimination models can predict the relative detectability of objects in those images, suggesting that these simpler models may be useful in some object detection applications. Here we replicate this result using a new, larger set of images. Fifteen images of a vehicle in an other-wise natural setting were altered to remove the vehicle and mixed with the original image in a proportion chosen to make the target neither perfectly recognizable nor unrecognizable. The target was also rotated about a vertical axis through its center and mixed with the background. Sixteen observers rated these 30 target images and the 15 background-only images for the presence of a vehicle. The likelihoods of the observer responses were computed from a Thurstone scaling model with the assumption that the detectabilities are proportional to the predictions of an image discrimination model. Three image discrimination models were used: a cortex transform model, a single channel model with a contrast sensitivity function filter, and the Root-Mean-Square (RMS) difference of the digital target and background-only images. As in the previous study, the cortex transform model performed best; the RMS difference predictor was second best; and last, but still a reasonable predictor, was the single channel model. Image discrimination models can predict the relative detectabilities of objects in natural backgrounds.

Ahumada, Albert J., Jr.↗