Search NASA⌕ Search

SEARCH · Search NASA

Results for “language”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

ESAC (EQ-SANS Assisting Chatbot): Application of large language models and retrieval-augmented generation for enhanced user experience at EQ-SANS

Neutron scattering experiments have played vital roles in exploring materials properties in the past decades. While user interfaces have been improved over time, neutron scattering experiments still require specific knowledge or training by an expert due to the complexity of such advanced instrumentation and the limited number of experiments each person may perform each year. This paper introduces an innovative chatbot application that leverages Large Language Models(LLM) and Retrieval-Augmented Generation (RAG) technologies to significantly enhance the user experience at the EQ-SANS, a small-angle neutron scattering instrument at the Spallation Neutron Source of Oak Ridge National Laboratory. Through a user-centric design approach, the EQ-SANS Assisting Chatbot (ESAC) serves as an interactive reference for users, thereby facilitating the use of the instrument by visiting scientists. By bridging the gap between the users of EQ-SANS and the control systems required to perform their experiments, the ESAC sets a new standard for interactive learning and support for the scientific community using large-scale scientific facilities.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Achieving GPT-4o level performance in astronomy with a specialized 8B-parameter large language model

AstroSage-Llama-3.1-8B is a domain-specialized natural-language AI assistant tailored for research in astronomy, astrophysics, cosmology, and astronomical instrumentation. Trained on the complete collection of astronomy-related arXiv papers from 2007 to 2024 along with millions of synthetically-generated question-answer pairs and other astronomical literature, AstroSage-Llama-3.1-8B demonstrates remarkable proficiency on a wide range of questions. AstroSage-Llama-3.1-8B scores 80.9% on the AstroMLab-1 benchmark, greatly outperforming all models—proprietary and open-weight—in the 8-billion parameter class, and performing on par with GPT-4o. This achievement demonstrates the potential of domain specialization in AI, suggesting that focused training can yield capabilities exceeding those of much larger, general-purpose models. AstroSage-Llama-3.1-8B is freely available, enabling widespread access to advanced AI capabilities for astronomical education and research.

AI assistant↗
DOE OSTI2025

Causal discovery from data assisted by large language models

Knowledge-driven discovery of novel materials necessitates the development of causal models for property emergence. While in the classical physical paradigm, the causal relationships are deduced based on physical principles or via experiment, the rapid accumulation of observational data necessitates learning causal relationships between dissimilar aspects of material structure and functionalities based on observations. For this, it is essential to integrate experimental data with prior domain knowledge. Here, we demonstrate this approach by combining high-resolution scanning transmission electron microscopy data with insights derived from large language models (LLMs). By applying ChatGPT to domain-specific literature, such as arXiv papers on ferroelectrics, and combining the obtained information with data-driven causal discovery, we construct adjacency matrices for directed acyclic graphs that map the causal relationships between structural, chemical, and polarization degrees of freedom in Sm-doped BiFeO 3 . This approach enables us to hypothesize how synthesis conditions influence material properties and guides experimental validation. Furthermore, the ultimate objective of this work is to develop a unified framework that integrates LLM-driven literature analysis with data-driven discovery, facilitating the precise engineering of ferroelectric materials by establishing clear connections between synthesis conditions and their resulting material properties.

Causal inference↗
DOE OSTI2024

ChatGPT and Other Large Language Models for Cybersecurity of Smart Grid Applications

Cybersecurity breaches targeting electrical substations constitute a significant threat to the integrity of the power grid, necessitating comprehensive defense and mitigation strategies. Any anomaly in information and communication technology (ICT) should be detected for secure communications between devices in digital substations. This paper proposes large language models (LLMs), e.g., ChatGPT, for the cybersecurity of IEC 61850-based communications. Multi-cast messages such as generic object oriented system events (GOOSE) and sampled values (SV) are used for case studies. The proposed LLM-based cybersecurity framework includes, for the first time, data pre-processing of communication systems and human-in-the-loop (HITL) training (considering the cybersecurity guidelines recommended by humans). The results show a comparative analysis of detected anomaly data carried out based on the performance evaluation metrics for different LLMs. A hardware-in-the-loop (HIL) testbed is used to generate and extract a dataset of IEC 61850 communications.

ChatGPT↗
DOE OSTI2026

From Rules to Reasoning: A Survey of Large Language Model-Based Approaches to Scientific Hypothesis and Idea Generation

Scientific hypothesis generation represents a fundamental challenge in contemporary research due to exponentially expanding literature volumes and increasing disciplinary specialization. Large language models (LLMs) have emerged as transformative tools for automated scientific discovery, moving beyond traditional rule-based and literature-mining approaches. Four paradigmatic approaches define current LLM-driven hypothesis generation: direct prompting and fine-tuning methods, knowledge-enhanced frameworks integrating retrieval-augmented generation (RAG), multi-agent collaborative systems simulating research teams, and reasoning-focused approaches implementing cognitive architectures. Domain-specific applications demonstrate statistical equivalence to human expert performance in social psychology, experimental validation in biomedical research, and near-expert quality in astronomy. Evaluation methodologies encompass human expert assessment, LLM-as-judge frameworks, and comprehensive benchmarking systems. Technical challenges include hallucination management, knowledge integration limitations, and balancing novelty with feasibility. Future directions emphasize hybrid neural-symbolic architectures and sophisticated human-AI collaboration models for responsible scientific discovery acceleration.

AI-driven discovery↗
DOE OSTI2024

Evaluating the Effectiveness of Retrieval-Augmented Large Language Models in Scientific Document Reasoning

Despite the dramatic progress in Large Language Model (LLM) development, LLMs often provide seemingly plausible but not factual information, often referred as hallucinations. Retrieval-augmented LLMs provide a non-parametric approach to solve these issues by retrieving relevant information from external data sources and augment the training process. These models helps to trace evidence from an externally provided knowledge base allowing the model predictions to be better interpreted and verified. In this work, we critically evaluate these models in their ability to perform in scientific document reasoning tasks. To this end, we tuned multiple such model variants with science-focused instructions and evaluated them on a scientific document reasoning benchmark for the usefulness of the retrieved document passages. Our findings suggest that models justify predictions in science tasks with fabricated evidence and leveraging scientific corpus as pretraining data does not alleviate the risk of evidence fabrication.

• Artificial intelligence (AI) / machine learning ↗
DOE OSTI2024

Improving Reliability of Large Language Models for Nuclear Power Plant Diagnostics [Poster]

Large Language Models (LLMs) struggle out of the box when answering factually about detailed questions, especially in domains that are sparsely represented in their training data. This causes hallucinations and reduces reliability making it difficult for them to be used in practice. This work shows that using RAG techniques can improve factual accuracy and reliability, allowing for the application of LLMs in specialized areas, even when those areas that aren’t extensively covered in their initial training.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Prompt Phrase Ordering Using Large Language Models in HPC: Evaluating Prompt Sensitivity

Large language models (LLMs) have demonstrated effective performance in domain-specific tasks, often requiring a well-designed prompt to guide their responses. However, optimizing the right prompt is challenging due to prompt sensitivity—the phenomenon where small changes in the prompt can lead to significant variations in performance. In this study, we evaluate prompt performance by examining all permutations of independent phrases to investigate prompt sensitivity and robustness. We used two datasets: the GSM8k dataset, which assesses mathematical reasoning, and a custom template prompt for summarizing database metadata. Our goal was to evaluate the performance across all permutations of a sequence of prompt phrases. The study was conducted using the llama3-instruct- 7B model hosted on Ollama, with computations parallelized in a high-performance computing environment. By comparing the average index of phrases in the best and worst-performing prompts, we found that the order of independent phrases within a prompt significantly impacts LLM performance. Additionally, we used Hamming distance to assess changes between phrase orderings, concluding that prompt modifications can dramatically affect scores, often by almost random chance. These findings support existing research on prompt sensitivity. We discuss the challenges of prompt optimization, noting that altering phrases in a successful prompt does not always result in another successful prompt.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Connecting Minds: AI Use Cases to Bridge Power Systems and Large Language Models for Practical Applications

Recent advances in artificial intelligence (AI) and development of large language models (LLMs) present the opportunity to develop a new generation of power systems applications. In contrast with early power system AI applications based on structured numerical data, LLMs offer unique capabilities to perform logical reasoning using text documents, unstructured data, and application programming interface (API) calls to computational software. This paper seeks to bridge the knowledge gap between power systems engineers and LLM developers through a crosscutting explanation of use cases, characteristics, requirements, practical considerations from the perspectives of both LLM capabilities and industry needs. Specific focus is given to applications that can be realistically deployed by electric utilities. After introducing the architecture of LLMs and unique challenges of the power systems domain, this paper proposes twenty representative LLM applications grouped into categories of 1) power system operations, 2) asset management, 3) system planning and analytics, and 4) energy management and protection systems. Five use cases are presented within each category with descriptions of the motivation, objectives, approaches, example inputs / outputs, and benefits of each use case.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Business Case Analysis for Artificial Intelligence-Large Language Model Technology Integration

AI-assisted processes are expected to enhance operational efficiency and improve decision-making, supporting the long-term economic viability of nuclear power plants. However, detailed business analyses of AI-generated cost savings are rarely performed. Given the recent industry interest in Large Language Model (LLM), the U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program has conducted a comprehensive business case analysis of LLM Artificial Intelligence (AI) implementation in nuclear plant engineering workflows. The research employed three complementary business case approaches to evaluate impact of an LLM, using three representative engineering processes as use-cases: Boric Acid Corrosion (BAC) Evaluations, Maintenance Rule Evaluations, and 10 CFR 50.59 Screenings. Through detailed workload analyses and structured interviews, the study quantified significant efficiency improvements ranging from 11% to 59% across these processes. The research further considers how these efficiency gains could translate into tangible reliability improvements through enhanced engineering capacity. Analysis of historical plant trip data indicates that enabling engineers to focus on proactive reliability activities could provide substantial financial benefits through avoided outages, potentially generating greater value than the direct efficiency improvements alone. By documenting successful applications, implementation challenges, and strategic opportunities, this research provides nuclear utilities with a practical framework for evaluating the value of AI technology to support long-term operations through advanced digital technologies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Data Driven Approach to Public Opinion Mining on Autonomous Vehicles: Sentiment Analysis of Social Media Comments Using Large Language Models

In the realm of online identity, social media has emerged as a rich and dynamic source of user-generated content, making it an invaluable resource for understanding public sentiment on a wide range of topics. Individuals often share their raw emotions and candid opinions on these platforms without fear of judgment or backlash. In this study, we conduct a sentiment analysis on user comments collected from various online platforms, with a specific focus on discussions surrounding autonomous vehicles. Leveraging the capabilities of large language models (LLMs), we classify each comment into one of five sentiment categories: Very Negative, Negative, Neutral, Positive, and Very Positive. Our approach demonstrates the effectiveness of LLMs in capturing nuanced contextual sentiment, offering a scalable and state-of-the-art alternative to traditional manual annotation methods. The results reveal key trends and insights into public perception, enabling a deeper understanding of how autonomous vehicle technologies are received by the online community. Our findings underscore the dynamic nature of public sentiment, which is shaped not only by advances in autonomous vehicle technology but also by contextual events such as regulatory developments, political adjustment and safety incidents.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Developing an Interactive OpenMP Book with Large Language Models

Abstract. This paper presents an approach to authoring a textbook titled Interactive OpenMP Programming with the assistance of Large Language Models (LLMs). The writing process utilized state-of-the-art LLMs, including Gemini Pro 1.5, Claude 3, and ChatGPT-4, to generate the initial structure and outline of the book, as well as the initial content for specific chapters. This content included detailed descriptions of individual OpenMP constructs and practical programming examples. The outline and content have then undergone extensive manual revisions to meet our book goals. In this paper, we report our findings about the capabilities and limitations of these LLMs. We address critical questions concerning the necessity of textbook resources and the effectiveness of LLMs in creating fundamental and practical programming content. Our findings suggest that while LLMs offer significant advantages in generating textbook content, they require careful integration with traditional educational methodologies to ensure depth, accuracy, and pedagogical effectiveness. The Interactive OpenMP Programming book is developed with the framework of Jupyter Book, enabling the execution of code within the book from the web browser, providing instant feedback and a dynamic learning experience that stands in contrast to traditional educational resources. The book represents a significant step towards modernizing programming education, offering insights into practical strategies for generating the textbook through advanced AI tools.

Large Language Model · OpenMP · Interactive Book ·↗
DOE OSTI2024

Reductive Analysis with Compiler-Guided Large Language Models for Input-Centric Code Optimizations

Input-centric program optimization aims to optimize code by considering the relations between program inputs and program behaviors. Despite its promise, a long-standing barrier for its adoption is the difficulty of automatically identifying critical features of complex inputs. This paper introduces a novel technique, reductive analysis through compiler-guided Large Language Models (LLMs), to solve the problem through a synergy between compilers and LLMs. It uses a reductive approach to overcome the scalability and other limitations of LLMs in program code analysis. The solution, for the first time, automates the identification of critical input features without heavy instrumentation or profiling, cutting the time needed for input identification by 44× (or 450× for local LLMs), reduced from 9.6 hours to 13 minutes (with remote LLMs) or 77 seconds (with local LLMs) on average, making input characterization possible to be integrated into the workflow of program compilations. Optimizations on those identified input features show similar or even better results than those identified by previous profiling-based methods, leading to optimizations that yield 92.6% accuracy in selecting the appropriate adaptive OpenMP parallelization decisions, and 20-30% performance improvement of serverless computing while reducing resource usage by 50-60%.

Input-Centric Optimization↗
DOE OSTI2025

Role of Uncertainty Quantification in the Explainability of Large Language Models for the Nuclear Industry

The meteoric rise of generative artificial intelligence (AI) large language models (LLMs) has created an opportunity to utilize them to increase efficiencies in a multitude of industries. While LLMs carry great potential to revolutionize the manner in which work is performed, numerous known deficiencies limit their utility, including the black box nature of the models, the stochastic nature of the response (i.e., presenting the same prompt multiple times results in different responses), and the potential for hallucination. Widespread adoption of LLMs in safety-critical industries such as nuclear will require some form of explainability to assure end users that the LLM’s response to a given query is valid. Model uncertainty is inherently linked to the concepts of trust and explainability, and can be used to identify situations in which the model is insufficiently certain about its answer. Although uncertainty is not enough in and of itself to determine the suitability of an answer—a model can be very certain of an inaccurate answer—it still provides valuable supporting information. Practical methodologies for gauging or quantifying the uncertainty in LLM outputs are presented herein, along with examples based on nuclear-specific prompts.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

System-Level Integration of Modular Language Models for Real-Time Risk Assessment in Third-Party Risk Management Systems

Large enterprises typically rely on dedicated teams to govern and implement security measures throughout their supply chains, ensuring compliance with enterprise security procedures. There is a significant reliance on Third-Party Risk Management (TPRM) platforms, which often require complete, highly structured information from potential vendors. The review and compliance assurance processes are time- and labor intensive, often requiring several rounds of review between the supply chain security risk management teams, business users, and potential vendors, leading to delays in the supply chain processing and consumer experience. Significant challenges in the risk management paradigm include handling unstructured data in various formats and providing real-time feedback to users to reduce the required review time. This paper presents a novel solution to these challenges. A modular multi-step system architecture is proposed using advances in language processing, specifically for unstructured responses and provides real-time feedback (i.e., 3 seconds) so that users can improve their responses before the TPSRM team review. This novel system architecture will increase information accuracy and significantly reduce time and labor during the review process.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2026

A Benchmarking Framework for Evaluating Large Language Model Capabilities in Nuclear Reactor Safety Applications

Large language models (LLMs) are increasingly capable of answering technical questions, synthesizing domain knowledge, and supporting engineering workflows. For nuclear science and engineering, these capabilities require careful, domain-specific evaluation before they can be credibly incorporated into safety-related activities, regulatory review, or technical decision support. This paper presents preliminary results from benchmarking framework for evaluating LLM capabilities in nuclear contexts. The framework is organized into three evaluation categories: nuclear fundamentals, general dual-use knowledge, and plant specific knowledge. These categories are intended to distinguish general nuclear engineering competence from broader technical reasoning and more context-dependent nuclear knowledge. Initial evaluations focus on nuclear fundamentals using questions representative of the knowledge expected of a nuclear professional engineer. Results indicate that contemporary frontier models perform at a high level and substantially exceed the performance of older model generations, with some models approaching saturation of the current benchmark. These findings suggest both the rapid improvement of LLM capabilities in specialized technical domains and the need for more discriminating evaluation methods. The paper presents the benchmark structure, preliminary model-comparison results, and ongoing work. This work supports development of verifiable, responsible, and safety-conscious methods for assessing AI systems in nuclear engineering applications.

21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗