Search NASA⌕ Search

SEARCH · Search NASA

Results for “large language model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

MechBERT: Language Models for Extracting Chemical and Property Relationships about Mechanical Stress and Strain

Language models are transforming materials-aware naturallanguage processing by enabling the extraction of dynamic, context-rich information from unstructured text, thus, moving beyond the limitations of traditional information-extraction methods. Moreover, small language models are on the rise because some of them can perform better than large language models (LLMs) when given domain-specific questionanswer tasks, especially about an application area that relies on a highly specialized vernacular, such as materials science. We therefore present a new class of MechBERT language models for understanding mechanical stress and strain in materials. These employ Bidirectional Encoder Representations for transformer (BERT) architectures. We showcase four MechBERT models, all of which were pretrained on a corpus of documents that are textually rich in chemicals and their stress–strain properties and were fine-tuned on question-answering tasks. We evaluated the level of performance of our models on domain-specific as well as general English-language question-answer tasks and also explored the influence of the size and type of BERT architectures on model performance. We find that our MechBERT models outperform BERT-based models of the same size and maintain relevancy better than much larger BERT-based models when tasked with domain-specific question-answering tasks within the stress–strain engineering sector. These small language models also enable much faster processing and require a much smaller fraction of data to pretrain them, affording them greater operational efficiency and energy sustainability than LLMs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Intern Poster

Large Language Models (LLMs) have skyrocketed in popularity after the release of ChatGPT in late 2022. Although LLMs are powerful tools, they can be subject to hallucinations, which is when an LLM (or any AI model) produces misleading/ nonsensical information. The objective is to determine if statistical methods can be used to detect hallucinations as an LLM generates its answer token by token (essentially word by word).

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Protein–Protein Interaction Networks Derived from Classical and Machine Learning-Based Natural Language Processing Tools

The study of protein-protein interactions (PPIs) provides insight into various biological mechanisms, including the binding of antibodies to antigens, enzymes to inhibitors or promoters, and receptors to ligands. Recent studies of PPIs have led to significant biological breakthroughs. For example, the study of PPIs involved in the human:SARS-CoV-2 viral infection mechanism aided in the development of the SARS-CoV-2 vaccines. Though several databases exist for the manual curation of PPI networks, text mining methods have been routinely demonstrated as useful alternatives for newly studied or understudied species where databases are incomplete. Here, the relationship extraction (RE) performance of several open-source classical text processing, machine learning (ML)-based natural language processing (NLP), and large language model (LLM)-based NLP tools were compared. Overall, our results indicated that networks derived from classical methods tend to have high true positive rates at the expense of having overconnected-networks, ML-based NLP methods have lower true positive rates but networks with the closest structures to the target network, and LLM-based NLP methods tend to exist in-between the two other approaches, with variable performances. Finally, the selection of a specific NLP approach should be tied to the needs of a study and text availability, as models varied in performance due to the amount of text provided.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Artificial intelligence to unlock real-world evidence in clinical oncology: A primer on recent advances

Purpose: Real world evidence is crucial to understanding the diffusion of new oncologic therapies, monitoring cancer outcomes, and detecting unexpected toxicities. In practice, real world evidence is challenging to collect rapidly and comprehensively, often requiring expensive and time-consuming manual case-finding and annotation of clinical text. In this Review, we summarise recent developments in the use of artificial intelligence to collect and analyze real world evidence in oncology. Methods: We performed a narrative review of the major current trends and recent literature in artificial intelligence applications in oncology. Results: Artificial intelligence (AI) approaches are increasingly used to efficiently phenotype patients and tumors at large scale. These tools also may provide novel biological insights and improve risk prediction through multimodal integration of radiographic, pathological, and genomic datasets. Custom language processing pipelines and large language models hold great promise for clinical prediction and phenotyping. Conclusions: Despite rapid advances, continued progress in computation, generalizability, interpretability, and reliability as well as prospective validation are needed to integrate AI approaches into routine clinical care and real-time monitoring of novel therapies.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2026

Expert evaluation of LLM world models: A high-T c superconductivity case study

Large Language Models (LLMs) show great promise as a powerful tool for scientific literature exploration. However, their effectiveness in providing scientifically accurate and comprehensive answers to complex questions within specialized domains remains an active area of research. Using the field of high-temperature cuprates as an exemplar, we evaluate the ability of LLM systems to understand the literature at the level of an expert. We construct an expert-curated database of 1,726 scientific papers that covers the history of the field, and a set of 67 expert-formulated questions that probe deep understanding of the literature. We then evaluate six different LLM-based systems for answering these questions, including both commercially available closed models and a custom retrieval-augmented generation (RAG) system capable of retrieving images alongside text. Experts then evaluate the answers of these systems against a rubric that assesses balanced perspectives, factual comprehensiveness, succinctness, and evidentiary support. Among the six systems, two using RAG on curated literature outperformed existing closed models across key metrics, particularly in providing comprehensive and well-supported answers. We discuss promising aspects of LLM performances as well as critical short-comings of all the models. The set of expert-formulated questions and the rubric will be valuable for assessing expert level performance of LLM based reasoning systems.

36 MATERIALS SCIENCE↗
DOE OSTI2026

What Is the Agent Doing? Visualizing Agentic AI Querying Workflows

We explore how visualizations can help users understand what an AI agent is doing as it builds and runs queries over data. As part of the LinkQ system, a natural language interface for querying knowledge graphs with a large language model (LLM), we designed two complementary views: A State Diagram that shows where the agent is within a larger workflow, and a Live Action Display that gives real-time updates about the agent's current task. In a study with 14 practitioners, we found that these visuals helped participants build stronger mental models of the agent's behavior while also increasing their confidence in the system. However, we also observed that users sometimes trusted incorrect outputs simply because the agent appeared to be doing the "right" thing. Our findings point to both the value and risk of visualizing agent behavior in interactive AI systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

eGridGPT: Trustworthy AI in the Control Room

This report outlines the synergy between human decision making and generative artificial intelligence (GenAI), where GenAI supports power system operators by analyzing procedures, suggesting actions, simulating scenarios with physics-based digital twins, and recommending optimal decisions. This report is the first research effort to apply large language models (LLMs), a type of GenAI, in the power grid control room. The authors describe the Electric Grid Generative Pretrained Transformer (eGridGPT), an LLM that virtually assists system operators. Developed with cybersecurity and regulatory requirements in mind, eGridGPT represents an opportunity to responsibly evolve control room technologies to meet the needs of a rapidly changing grid. As an innovative concept, eGridGPT seeks to spearhead productive discussions about the advanced technologies in the control room of the future amid the transition to clean energy.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRSDate not supplied

Retrieval-Augmented Generation and LLM Agents for Biomimicry Design Solutions

We present BIDARA, a Bio-Inspired Design And Research Assistant, to address the complexity of biomimicry – the practice of designing modern-day engineering solutions inspired by biological phenomena. Large Language Models (LLMs) have been shown to act as sufficient general purpose task solvers, but they often hallucinate and fail in regimes that require domain-specific and up-to-date knowledge. We integrate Retrieval-Augmented Generation (RAG) and Reasoning-and-Action agents to aid LLMs in avoiding hallucination and utilizing updated knowledge during generation of biomimetic design solutions. We find that incorporating RAG increases the feasibility of the design solutions in both prompting and agent settings, and we use these findings to guide our ongoing work. To the extent of our knowledge, this is the first work that integrates and evaluates Retrieval-Augmented Generation within LLM-generated biomimetic design solutions.

biomimicry↗
NASA NTRSDate not supplied

Retrieval-Augmented Generation and LLM Agents for Biomimicry Design Solutions

We present BIDARA, a Bio-Inspired Design And Research Assistant, to address the complexity of biomimicry – the practice of designing modern-day engineering solutions inspired by biological phenomena. Large Language Models (LLMs) have been shown to act as sufficient general purpose task solvers, but they often hallucinate and fail in regimes that require domain-specific and up-to-date knowledge. We integrate Retrieval-Augmented Generation (RAG) and Reasoning-and-Action agents to aid LLMs in avoiding hallucination and utilizing updated knowledge during generation of biomimetic design solutions. We find that incorporating RAG increases the feasibility of the design solutions in both prompting and agent settings, and we use these findings to guide our ongoing work. To the extent of our knowledge, this is the first work that integrates and evaluates Retrieval-Augmented Generation within LLM-generated biomimetic design solutions.

biomimicry↗
DOE OSTI2026

Performance-Aligned LLMs for Generating Fast HPC Code

Optimizing scientific software is a difficult task because codebases are often large and complex, and performance can depend upon several factors including the algorithm, its implementation, and hardware among others. Causes of poor performance can originate from disparate sources and be difficult to diagnose. Recent years have seen a multitude of work that use large language models (LLMs) to assist in software development tasks. However, these tools are trained to model the distribution of code as text, and are not specifically designed to understand performance aspects of code. In this work, we introduce a reinforcement learning based methodology to align the outputs of code LLMs with performance. This allows us to build upon the current code modeling capabilities of LLMs and extend them to generate better performing code. Here, we demonstrate that our fine-tuned model improves the expected speedup of generated code over base models for a set of benchmark tasks from 0.9 to 1.6 for serial code and 1.9 to 4.5 for OpenMP parallel code.

Computer science↗
DOE OSTI2025

Unlocking Scholarly Insights: Leveraging Machine Learning Approaches for Citation Analysis and Intent Classification

Publicly funded organizations, notably institutions like the Los Alamos National Laboratory (LANL), are deeply vested in acquiring robust productivity metrics to gauge the entirety of their research output. Motivated by the imperative to enhance institutional productivity assessment, this study investigates the utilization of Large Language Models (LLM) such as BERT-based models, as well as local LLaMa-30b-instruct and Mixtral-8x7b-instruct architectures for classifying type of URL referenced resources in academic papers such as software, dataset, as well as authorship intent. Challenges in discerning resource types from context are highlighted, along with the potential of BERT and LLMs to address these challenges. Through comprehensive analysis, this research unveils a notable surge in documents featuring URL citations, indicative of the escalating importance of digital resources in scholarly publications. Moreover, citations to datasets and software demonstrate consistent growth over time, underscoring their increasing significance. Our findings also reveal that LANL authors contribute substantially to accessible science, comprising about 10% of dataset and software mentions in LANL

Large Language Models, BERT, citation classificati↗
DOE OSTI2025

LLM-Based Adaptive Distribution Voltage Regulation Under Frequent Topology Changes: An In-Context MPC Framework

This paper proposes a large language model (LLM) based adaptive inverter control for distribution voltage regulation under frequent topology changes. We leverage the ability of the LLM to perform in-context learning and create a topology-adaptive surrogate model for power flow calculation. The surrogate model is then integrated with a long short-term memory-based load forecaster and a model predictive control (MPC) scheme to achieve the optimal inverter control that adapts to frequent topology changes. Unlike many existing works that assume fixed-topology grids or require the knowledge of all possible topologies when training a model, the proposed in-context MPC method tackles the distribution voltage control problem under various topologies and adapts to unknown topologies with limited data requirement for fine-tuning. The effectiveness of our method is demonstrated on a modified IEEE 123-bus test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Trustworthiness and Trust: Identifying Factors that Drive Successful Human-AI Interaction in Nuclear Power Plant Applications

Emerging technologies such as artificial intelligence (AI) and machine learning (ML) are rapidly evolving and considered a promising tool for efficient and continued safe operations of the U.S. nuclear power plants (NPPs). Emerging AI techniques like large language models (LLMs) are one such technology that may support personnel at existing NPPs perform work more efficiently. For example, operators may query the current operational status of a power plant via a chat interface leveraging LLMs to access plant-related information in an interactive manner rather than manually collecting various sensor data for tasks such as surveillances or completing work orders. This is a fundamental shift in the way operators currently perform their tasks today. The literature of human-automation interaction indicates that trust is a crucial factor that drives successful interaction between a human operator and an automated system, like an AI-infused NPP application. This work presents the results of a literature review on key factors that relate to trust in AI/LLM technologies for NPP applications. The relevant literature of human factors and cognitive engineering has identified various factors related to trust including trustworthiness, performance characteristics, operator skill and perceived risk. This preliminary literature review will guide development and evaluation of models involving the identified factors influencing trust in AI and develop a framework for human-centered design for interface between humans and AI. By addressing trust, this work supports developing a technical basis for designing key characteristics of AI/LLM to support calibrated trust, which will ultimately support wide-scale adoption of AI/LLM technologies, as well as ensure safe, effective, and reliable use.

99 - GENERAL AND MISCELLANEOUS↗
NASA NTRSDate not supplied

Requirement Discovery Using Embedded Knowledge Graph with ChatGPT

The field of Advanced Air Mobility (AAM) is witnessing a transformation with innovations such as electric aircraft and increasingly automated airspace operations. Within AAM, the Urban Air Mobility (UAM) concept focuses on providing air-taxi services in densely populated urban areas. This research introduces the utilization of Large Language Models (LLMs), such as OpenAI's GPT-4, to enhance the UAM Requirement discovery process. This study explores two distinct approaches to leverage LLMs in the context of UAM Requirement discovery. The first approach evaluates the LLM's ability to provide responses without relying on additional outside systems, such as a relational or graph database. Instead, a vector store provides relevant information to the LLM based on the user’s question, a process known as Retrieval Augmented Generation (RAG). The second approach integrates the LLM with a graph database. The LLM acts as an intermediary between the user and the graph database, translating user questions into cypher queries for the database and database responses into human-readable answers for the user. Our team implemented and tested both solutions to analyze requirements within a UAM dataset. This paper will talk about our approaches, implementations, and findings related to both approaches.

systems engineering↗
DOE OSTI2025

ON THE EFFECTIVENESS OF LLMS IN UNIT TEST GENERATION FOR STRUCTURED TEXT PROGRAMS

The reliability of industrial automation systems heavily depends on the correctness of Programmable Logic Controller (PLC) programs, which are often written in Structured Text (ST). While Large Language Models (LLMs) have shown promise in automating test generation for mainstream programming languages, their effectiveness for the syntactically strict ST language remains underexplored. This thesis presents a systematic empirical evaluation of three state-of-the-art LLMs—GPT-4o, Gemini 2.5 Pro, and Claude Sonnet 4.5—for generating ST unit tests. We examine three prompting strategies: Natural Language (NL), Code Language (CL), and Chain-of-Thought (CoT), across a curated set of 11 ST function blocks. The quality of the generated tests is assessed using Compilation Success Rate (CSR), Statement Coverage (SC), and Branch Coverage (BC). In the zero-shot setting, Claude Sonnet 4.5 achieves the highest CSR, while Gemini 2.5 Pro consistently delivers the best statement and branch coverage, particularly under CL prompts. By incorporating a one-shot CL prompt, all models exhibit substantial improvements—most notably GPT-4o, whose CSR increases from 45.45% to 90.91%, with substantial gains in both SC and BC. To further contextualize these findings, we compare GPT-4o’s one-shot results with PLCAutoTester, a state-ofthe- art ST unit test generation tool, on an additional benchmark dataset. While LLMgenerated tests approach competitive coverage levels, PLCAutoTester maintains significantly higher and more stable coverage across programs. This study provides the first comprehensive benchmark of modern LLMs for ST unit testing, highlighting their strengths, limitations, and improvements through one-shot prompting, and positioning their performance relative to specialized automated testing tools in industrial automation.

42 ENGINEERING↗
NASA NTRSDate not supplied

Requirement Discovery Using Embedded Knowledge Graph With ChatGPT

The field of Advanced Air Mobility (AAM) is witnessing a transformation with innovations such as electric aircraft and increasingly automated airspace operations. Within AAM, the Urban Air Mobility (UAM) con-cept focuses on providing air-taxi services in densely populated urban areas. This research introduces the utilization of Large Language Models (LLMs), such as OpenAI's GPT-4, to enhance the UAM Requirement discovery process. This study explores two distinct approaches to leverage LLMs in the context of UAM Requirement discovery. The first approach evaluates the LLM's ability to provide responses without relying on additional outside systems, such as a relational or graph database. Instead, a vector store provides relevant information to the LLM based on the user’s question, a process known as Retrieval Augmented Generation (RAG). The second approach integrates the LLM with a graph database. The LLM acts as an intermediary between the user and the graph database, translating user questions into cypher queries for the database and database responses into human-readable answers for the user. Our team implemented and tested both solutions to analyze require-ments within a UAM dataset. This paper will talk about our approaches, implementations, and findings related to both approaches.

systems engineering↗
NASA NTRSDate not supplied

Requirement Discovery Using Embedded Knowledge Graph With ChatGPT - Poster

The field of Advanced Air Mobility (AAM) is witnessing a transformation with innovations such as electric aircraft and increasingly automated airspace operations. Within AAM, the Urban Air Mobility (UAM) con-cept focuses on providing air-taxi services in densely populated urban areas. This research introduces the utilization of Large Language Models (LLMs), such as OpenAI's GPT-4, to enhance the UAM Requirement discovery process. This study explores two distinct approaches to leverage LLMs in the context of UAM Requirement discovery. The first approach evaluates the LLM's ability to provide responses without relying on additional outside systems, such as a relational or graph database. Instead, a vector store provides relevant information to the LLM based on the user’s question, a process known as Retrieval Augmented Generation (RAG). The second approach integrates the LLM with a graph database. The LLM acts as an intermediary between the user and the graph database, translating user questions into cypher queries for the database and database responses into human-readable answers for the user. Our team implemented and tested both solutions to analyze require-ments within a UAM dataset. This paper will talk about our approaches, implementations, and findings related to both approaches.

systems engineering↗
DOE OSTI2025

Lessons Learned from AskGDR: Usage and Impact Analysis of the Geothermal Data Repository's AI Research Assistant: Preprint

In October of 2024, the Department of Energy's (DOE) Geothermal Data Repository (GDR) team officially launched AskGDR, an AI research assistant resulting from the integration of a Large Language Model (LLM) with the metadata and supporting documents associated with GDR datasets. AskGDR allows GDR users to ask deeper questions about the origin of datasets, the methods used to collect them, and the findings they help support. Using Retrieval Augmented Generation (RAG), AskGDR can be used to summarize findings spread across dozens of papers and technical reports or to extract relevant information describing a single data field. However, generative AI is experimental. The National Renewable Energy Laboratory (NREL) has been collecting metrics on AskGDR and documenting lessons learned during its deployment. This paper will outline the efficacy and impact of AskGDR through analysis of its use, operating costs, number and types of questions asked, and the quality of answers provided.

15 GEOTHERMAL ENERGY↗