SEARCH · Search NASA

Results for “transformer models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2024

The Case for Co-Designing Model Architectures with Hardware

While GPUs are responsible for training the vast majority of state-of-the-art deep learning models, the implications of their architecture are often overlooked when designing new deep learning (DL) models. As a consequence, modifying a DL model to be more amenable to the target hardware can significantly improve the runtime performance of DL training and inference. In this paper, we provide a set of guidelines for users to maximize the runtime performance of their transformer models. These guidelines have been created by carefully considering the impact of various model hyperparameters controlling model shape on the efficiency of the underlying computation kernels executed on the GPU. We find the throughput of models with “efficient” model shapes is up to 39% higher while preserving accuracy compared to models with a similar number of parameters but with unoptimized shapes.

Yin, Junqi↗

NASA NTRSDate not supplied

Natural Language Processing Analysis of Notices to Airmen for Air Traffic Management Optimization

With new emerging technologies in the field of NLP, we explore their applications to digitize and analyze heritage Air Traffic Management (ATM) documents for planning and optimizing airspace operations. Specifically, this research focuses on harvesting semi-structured or un-structured information contained in Notices to Airmen (NOTAMs). Using NLP and other advanced data analytics, we will construct a data-driven framework which facilitates finding language patterns and the use of pretrained language models for classification and extraction of useful airspace constraints and restrictions. These may lead to tools that assist airspace users in understanding the constraints more efficiently, contributing to better route planning and safer execution. This paper explores three workflows entailing different NLP tasks. First, unsupervised techniques like word embedding and topic modeling are used for pattern finding and document classification. Second, a dataset is created by extracting information from the semi-structured NOTAM format as metadata for categorizing, visualizing, and extracting key entities driving NOTAM content. Third, modern pre-built deep learning based transformer models such as BERT, RoBERTa, and XLNet are evaluated on the question answering task, an even more robust approach to information extraction, as well as their respective fine-tuning tasks. In this work we include various performance metrics for the trained models to evaluate both accuracy and precision and we show that the models can be generalized for their respective tasks. The research work developed shows promise in uncovering trends in digital NOTAMs in the NAS and also offers a new framework for digitizing and inferring insights from free-form legacy NOTAMs, that are yet to be digitized.

Natural Language Processing↗

NASA NTRSDate not supplied

Natural Language Processing (NLP) Analysis of NOTAMs for Air Traffic Management Optimization

Natural Language Processing↗

DOE OSTI2026

Xanthos-Lake Model Source Code

This repository contains the source code for Xanthos-Lake, a lake-modeling extension of the Xanthos framework that introduces a coupled lake component comprising the Xanthos-Lake Snow and Ice Model (xLSIM) and the Xanthos-Lake Water Balance Model (xLWBM). xLSIM is a basin-aware machine-learning model for lake snow, ice, and thermal conditions. It predicts monthly lake ice thickness, snow depth, snow-cover fraction, mixing-layer temperature, and lake ice fraction from meteorological forcing and lake surface-area information. It uses sequence-based deep-learning architectures, including Transformer and hybrid Long Short-Term Memory–Transformer (LSTM–Transformer) models, together with seasonal encoding, multi-lake learning, physical masking, and basin-level cryospheric and non-cryospheric classification. The training workflow uses Ray for scalable execution and includes optional Ray Tune hyperparameter optimization. Model predictions, observations, diagnostics, and feature-importance outputs are written in NetCDF. xLWBM is the water-balance component of the new lake framework. It simulates monthly lake storage, surface area, evaporation, inflow, outflow, and lake–groundwater exchange. It combines physical water-balance equations with calibrated bathymetric relationships, weir-based outlet flow, modified Penman open-water evaporation, groundwater head relaxation, Penman–Monteith snow and ice sublimation, and snow, ice, and thermal conditions supplied by xLSIM. The model calibrates lake parameters against satellite-derived surface-area data, using evaporation-based calibration where surface-area data are unavailable, and supports small, medium, and large lake classes. For large lakes, xLWBM is integrated with the managed-routing workflow so that lake storage and outflow interact directly with downstream river routing and reservoir operations. Together, xLSIM and xLWBM provide Xanthos with a coupled lake-modeling capability. xLSIM supplies the snow, ice, and thermal conditions that affect lake evaporation and snow- and ice-related water exchanges, while xLWBM translates those conditions into dynamic lake storage, surface area, evaporation, and discharge. In return, xLWBM supplies evolving lake surface area to xLSIM. This coupling enables Xanthos to represent lakes as active hydrologic components within basin-scale water-availability and routing simulations.

Machine Learning↗

NASA NTRS1975

Coupling of Helmholtz resonators to improve acoustic liners for turbofan engines at low frequency

An analytical and test program was conducted to evaluate means for increasing the effectiveness of low frequency sound absorbing liners for aircraft turbine engines. Three schemes for coupling low frequency absorber elements were considered. These schemes were analytically modeled and their impedance was predicted over a frequency range of 50 to 1,000 Hz. An optimum and two off-optimum designs of the most promising, a parallel coupled scheme, were fabricated and tested in a flow duct facility. Impedance measurements were in good agreement with predicted values and validated the procedure used to transform modeled parameters to hardware designs. Measurements of attenuation for panels of coupled resonators were consistent with predictions based on measured impedance. All coupled resonator panels tested showed an increase in peak attenuation of about 50% and an increase in attenuation bandwidth of one one-third octave band over that measured for an uncoupled panel. These attenuation characteristics equate to about 35% greater reduction in source perceived noise level (PNL), relative to the uncoupled panel, or a reduction in treatment length of about 24% for constant PNL reduction. The increased effectiveness of the coupled resonator concept for attenuation of low frequency broad spectrum noise is demonstrated.

Dean, L. W.↗

DOE OSTI2026

Attention-based explainability for structure–property relationships

Machine learning methods are emerging as a universal paradigm for constructing correlative structure–property relationships in materials science based on multimodal characterization. However, this necessitates the development of methods for the physical interpretability of the resulting correlative models. Here, we demonstrate the potential of attention-based neural networks for revealing structure–property relationships and the underlying physical mechanisms, using the ferroelectric properties of PbTiO3 thin films as a case study. Through the analysis of attention scores, we disentangle the influence of distinct domain patterns on the polarization switching process. The attention-based Transformer model is explored both as a direct interpretability tool and as a surrogate for explaining representations learned via unsupervised machine learning, enabling the identification of physically grounded correlations. We compare attention-derived interpretability scores with classical SHapley Additive exPlanations analysis and show that, in contrast to applications in natural language processing, attention mechanisms in materials science exhibit high efficiency in highlighting meaningful structural features.

Slautin, Boris [Independent Researcher]↗

DOE OSTI2025

Microreactor Automated Control System - Digital Twin Models and Advanced Control Systems Updates

Automation of control systems is expected to be important in the economic and safe operation of microreactors. Therefore, there is a need to develop and demonstrate automated control for microreactors, along with the development of testbeds for this purpose. This report provides updates on the status of a nonnuclear microreactor automated control system (MACS)—a real-time, hardware-in-the-loop testbed for non-nuclear testing of microreactor control system automation. A real-time hardware-in-the-loop testbed incorporates the realistic dynamics of physical systems into control system development and testing. The collaborative effort between Oak Ridge National Laboratory (ORNL) and Idaho National Laboratory (INL) resulted in the development of a prototypic microreactor plant-level digital twin that includes the reactor and a balance of plant system. Advanced control strategies were incorporated to demonstrate testing of control automation solutions. The gRPC communication protocol, which was implemented in the hardware-in-the-loop testbed by INL, was coupled to a digital twin model developed using the TRANsient Simulation Framework of Reconfigurable Models (TRANSFORM) library in Modelica. This digital twin simulation was tested with the ViBRANT hardware for realistic feedback and visual representation of control action in real time. A modular Python client structure was developed to manage functional mock-up unit-based simulation and real-time gRPC communication. Hardware-in-the-loop testing indicated that the modeled reactor—a natural-convection, molten-salt coolant loop configuration—responds well to control of drum positioning for modulation of reactor core power, as well as system-level control and downstream demand changes. Ongoing research is focused on integrating additional control algorithms that utilize data from newly included sensors within the MACS hardware testbed, as well as demonstrating and assessing the performance of the different automated control algorithms on multiple additional operational scenarios.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

NASA NTRS2005

Non-Gaussian Analysis of Turbulent Boundary Layer Fluctuating Pressure on Aircraft Skin Panels

The purpose of the study is to investigate the probability density function (PDF) of turbulent boundary layer fluctuating pressures measured on the outer sidewall of a supersonic transport aircraft and to approximate these PDFs by analytical models. Experimental flight results show that the fluctuating pressure PDFs differ from the Gaussian distribution even for standard smooth surface conditions. The PDF tails are wider and longer than those of the Gaussian model. For pressure fluctuations in front of forward-facing step discontinuities, deviations from the Gaussian model are more significant and the PDFs become asymmetrical. There is a certain spatial pattern of the skewness and kurtosis behavior depending on the distance upstream from the step. All characteristics related to non-Gaussian behavior are highly dependent upon the distance from the step and the step height, less dependent on aircraft speed, and not dependent on the fuselage location. A Hermite polynomial transform model and a piecewise-Gaussian model fit the flight data well both for the smooth and stepped conditions. The piecewise-Gaussian approximation can be additionally regarded for convenience in usage after the model is constructed.

Rizzi, Stephen A.↗

DOE OSTI2025

Sensitivity-based voltage constraints for optimal power flow in low-voltage distribution feeders

The optimal power flow (OPF) problem for distribution systems can include network details down to the low-voltage (LV) points of interconnection of individual customers. This paper addresses the implementation of voltage magnitude constraints, and sets forth a practicable approach for capturing the effects on voltage from the switching behavior of loads (e.g., heat pumps, air conditioners, water heaters, or pool pumps) and from the variability of renewable generation (e.g., rooftop solar). The proposed method adjusts the OPF voltage constraints based on forecasts of load and generation upper and lower bounds, in conjunction with sensitivity factors derived from the power flow equations. An illustrative OPF formulation is also provided, which incorporates transformer models that include core loss. We demonstrate that accurate modeling of these LV network components is critical to avoid voltage violations at customer points of interconnection. Furthermore, the ideas are validated through numerical case studies on a realistic distribution feeder.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Minimum entropy filtering for a single output non-Gaussian stochastic system using state transformation

This paper presents a novel filter design for the single-output stochastic non-linear systems subjected to non-Gaussian noises and the proposed assumptions. Based on a state transformation, the unmeasurable states of the systems can be estimated where non-linear terms in the systems have been eliminated. It has been shown that the estimation error is linearly dynamical regarding to the presented vector-valued filter gain which can be optimised by minimising the entropy-based performance criterion. In addition, the convergence of the presented algorithm is analysed in mean-square sense and a numerical example is given to verify the effectiveness of the presented filtering algorithm. Meanwhile, the extended Kalman filter, unscented particle filter and minimum entropy filter are given for the comparisons of the filtering performance. Following the presented framework, some extensions of the presented filtering algorithm are discussed to indicate the flexibility of the filter design. The contribution of this paper can be summarised as establishing a novel minimum entropy filtering framework which consists of model transformation, entropy optimisation and convergence analysis.

42 ENGINEERING↗

DOE OSTI2025

SigTime: Learning and Visually Explaining Time Series Signatures

Understanding and distinguishing temporal patterns in time series data is essential for scientific discovery and decision-making. For example, in biomedical research, uncovering meaningful patterns in physiological signals can improve diagnosis, risk assessment, and patient outcomes. However, existing methods for time series pattern discovery face major challenges, including high computational complexity, limited interpretability, and difficulty in capturing meaningful temporal structures. Here, to address these gaps, we introduce a novel learning framework that jointly trains two Transformer models using complementary time series representations: shapelet-based representations to capture localized temporal structures and traditional feature engineering to encode statistical properties. The learned shapelets serve as interpretable signatures that differentiate time series across classification labels. Additionally, we develop a visual analytics system—SigTime—with coordinated views to facilitate exploration of time series signatures from multiple perspectives, aiding in useful insights generation. We quantitatively evaluate our learning framework on eight publicly available datasets and one proprietary clinical dataset. Additionally, we demonstrate the effectiveness of our system through two usage scenarios along with the domain experts: one involving public ECG data and the other focused on preterm labor analysis.

97 MATHEMATICS AND COMPUTING↗

NASA NTRSDate not supplied

Harnessing Artificial Intelligence for Medical Diagnosis and Treatment During Space Exploration Missions

From May 8th to June 9th, 2023, I had the opportunity to participate in an experiential learning experience at Johnson Space Center in Houston, TX with Exploration Medical Capability (ExMC), an element of the NASA Human Research Program. During this research experience, I was not only able to work on the above titled research project, but also gain an immense exposure to the field of aerospace medicine, make numerous connections within the field, tour NASA facilities, as well as travel to the Aerospace Medical Association Annual Conference (AsMA) in New Orleans. To briefly introduce my project, it is well understood that the medical capabilities available to crew medical officers (CMOs) on the International Space Station will be different than the capabilities available and needed during deep space exploration missions to the Moon, Mars, and beyond. Ground support is particularly limited due to distance, communication delays (or lack of communication), and lack of resupply. Therefore, to support medical care by CMOs on these missions, robust clinical decision support systems (CDSSs) must be designed. The recent publication and public launch of generative artificial intelligence (AI) tools based upon large language models (LLM) such as ChatGPT provides the opportunity to create a smart assistant for onboard triage, diagnosis, and treatment of medical conditions. Ultimately, the overall purpose of the project was to research what AI tools currently exist or are in development, and to see how they might be implemented onboard during exploration class spaceflights of the future. The ExMC element is actively developing several tools to be used in preparation for and during deep space exploration missions. One of those tools, known as IMPACT, is a probabilistic risk assessment model which can be used to propose a desired medical system (based on mass and volume) and suggest the clinical outcomes likely to occur for a design reference mission (DRM). The group recently presented the IMPACT model and a DRM of interest titled “Modified Long Duration Lunar Orbital and Lunar Surface” (mLDLOLS) at the recent AsMA conference. The mLDLOLS mock mission is a 9 month and 6-day deep space exploration mission consisting of time in Moon’s orbit (3 months on the Gateway space station), on the lunar surface (3 months within habitat), and another 3 months on Gateway before return to Earth. For this DRM, IMPACT ultimately outlined a preferred medical system that was then associated with medical conditions considered to be most likely based on frequency, most likely to cause astronaut task time loss (TTL), most likely to cause return to definitive care (RTDC), and most likely cause loss of crew life (LOCL). IMPACT also highlighted the medical capabilities/skills that would be required to care for those medical conditions, such as performing a history of present illness or musculoskeletal exam with ultrasound. The primary objective of the project was to perform a survey of the AI tools and systems applicable to the conditions outlined for the proposed mLDLOLS mission. Using PubMed (including most relevant MeSH terms) and Google Scholar, we then created a robust annotated bibliography organized by condition. The 56-page and over 500 reference annotated bibliography was subsequently used to create a review outline that would become the basis for drafting of a future publication. For the review outline, we took those medical conditions researched within the annotated bibliography (condition-based approach) and deployed a systems-based approach, combining those medical conditions and related tools into ten categories. These categories included general/all-purpose CDSSs, tools to diagnose or manage respiratory, dermatologic, neurologic, auditory and vestibular, ophthalmic, musculoskeletal, infection-associated, and gynecologic conditions, as well as tools that could be deployed in the setting of trauma/emergency. With the completion of the 30-page outline, we then began drafting the review paper. To conclude the research experience, I presented the findings from our survey to the ExMC Clinical and Science team. With these objectives, I ultimately learned about the number of AI tools that exist today to assist medical professionals with the triage, diagnosis, and management of several medical conditions. These tools can span from chatbot assistants to help triage knee pain to vision transformer models that can identify ophthalmic conditions based on ocular surface images captured with a cell phone. We also highlighted the current gaps that exist in the literature alongside the advancements that are needed to make the desired CDSS for deep space exploration missions. With this experience, I certainly confirmed an existing career goal and identified several additional skills needed to become an aerospace medical doctor including knowledge of critical care in an extreme medicine setting, aerospace engineering and human integration systems, artificial intelligence, machine learning, and risk models. I also identified numerous transferable skills for this career goal including the basic knowledge of medicine (MD), deployment of the scientific method for critical thought about new scientific questions (PhD), review of published literature, including creating an annotated bibliography (PhD), as well as detailed scientific writing (PhD). The results of my research will likely guide the design of an all-encompassing onboard medical assistant for use during deep space exploration missions of the future. I plan on sharing the outcomes from this experience with my peers at a student seminar in the Fall semester on August 30th. During the seminar, I will detail the project, my experience at NASA and AsMA, as well as offer best practice guidelines for students entertaining similar experiences or careers. In conclusion, I would like to thank the WVU School of Medicine, Research and Graduate Education office, as well as NASA ExMC for the unwavering support of this life-changing experience.

Ryan A. Lacinski↗

DOE OSTI2026

Energy–Performance Trade-offs in Privacy-Preserving Federated Learning on SmartNIC-Enabled HPC Systems

Federated learning (FL) is increasingly deployed on accelerator-rich high-performance computing (HPC) systems, yet the system-level energy cost of privacy-aware FL remains poorly understood, particularly across heterogeneous networking and server-placement options. We present a measurement-driven study of energy–performance trade-offs for FL on GH200-class nodes across three deployment configurations: CPU-Ethernet, CPU-InfiniBand (RDMA-capable), and a DPU-hosted FL server over InfiniBand using a BlueField-3 SmartNIC/DPU. Using NVIDIA FLARE (NVFLARE), we align node-level power telemetry with per-round timing extracted from NVFLARE logs to quantify time-to-solution (TTS), energy-to-solution (ETS), energy-delay product (EDP), and synchronization behavior for three transformer models (ALBERT, DistilBERT, BERT), trained with and without differential privacy (DP). We find that interconnect choice is the dominant driver of runtime and energy: host-managed InfiniBand consistently reduces communication overhead versus Ethernet, yielding lower TTS/ETS/EDP. In contrast, in our NVFLARE deployment, placing the FL server on the DPU does not consistently match CPU-InfiniBand performance and can be slower—especially for larger models—highlighting that server placement alone is not sufficient to guarantee end-to-end gains. Finally, under our fixed-round protocol, DP increases per-round cost and runtime variance; ETS increases largely in proportion to TTS because average node power remains relatively stable across configurations.

Kotevska, Olivera [ORNL] (ORCID:0000000316772243)↗

DOE OSTI2026

Secondary-Side Active Rectifier Synchronization and Control in LCC-LC Compensated Inductive Power Transfer Systems

This paper proposes a communication-less synchronization and output power control strategy for LCC-LC compensated inductive power transfer systems with a secondaryside active rectifier. By replacing the passive rectifier with an active bridge, the proposed architecture eliminates the need for a separate secondary-side DC-DC converter and enables direct regulation of battery power. The key challenge in such a system is achieving robust synchronization of the secondary active bridge with the primary inverter without access to primary-side signals. To address this issue, the secondary-side variables are analyzed and the current through the secondary series inductor is identified as a load-independent synchronization variable with a fixed phase relationship to the primary excitation. A fixed-frequency second order generalized (SOGI)-based phase locked loop (PLL) is used to estimate the switching phase and frequency, and a phase-shift-based output power controller is developed using a phasor-transformer model of the active bridges. A modified control formulation is further introduced to remove phase-dependent loop-gain variation and simplify controller design. Simulation results validate the proposed method under both fixed and varying switching-frequency conditions, demonstrating successful frequency tracking from 82 kHz to 88 kHz and accurate output power regulation over a wide operating range. The proposed method offers a compact and effective solution for high-power IPT systems by enabling secondary-side synchronization and control without wireless communication or an additional DC-DC conversion stage.

Fernandes, Arnold Anthony [ORNL] (ORCID:0009000631↗

DOE OSTI2024

CoVTransformer

The code is a transformer model to forecast SARS-CoV-2 lineage frequencies in the future.

Feng, Yinan↗

DOE OSTI2026

AI4HE: A Python Library for Materials Discovery

Machine learning library for developing scientific transformer models, particularly chemical language models, at scale

Salij, Andrew↗

DOE OSTI2026

Transform-Your-World

Machine learning library for developing scientific transformer models, particularly chemical language models, at scale

Salij, Andrew↗

NASA NTRS1987

Automated rectification and geocoding of SAR imagery

An automated post-processing system has been developed for rectification and geocoding of SAR (Synthetic Aperture Radar) imagery. The system uses as input a raw uncorrected image from the operational SAR correlator, and produces as a standard output a rectified and geocoded product. The accurate geolocation of SAR image pixels is provided by a spatial transformation model which maps the slant range-azimuth SAR image pixels into their location on a prespecified map grid. This model predicts the geodetic location of each pixel by utilizing: the sensor platform position; a geoid model; the parameters of the data collection system and the processing parameters used in the SAR correlator. Based on their geodetic locations, the pixels are mapped by using the desired cartographic projection equations. This rectification and geocoding technique has been tested with Seasat and SIR-B images. The test results demonstrate absolute location uncertainty of less than 50 m and relative distortion (scale factor and skew) of less than 0.1 percent relative to local variations from the assumed geoid.

Kwok, R.↗