Large Language Model (LLM) Driven Document Clustering: Improving Real time Security Intelligence Extraction and Threat Analysis
Explore the source record for details and available documents.
Computer science↗
SEARCH · Search NASA
Results for “Large Language Models”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
C-LoRA: Contextual Low-Rank Adaptation for Uncertainty Estimation in Large Language Models
Explore the source record for details and available documents.
97 MATHEMATICS AND COMPUTING↗
Improving Text Classification with Large Language Model-Based Data Augmentation
Large Language Models (LLMs) such as ChatGPT possess advanced capabilities in understanding and generating text. These capabilities enable ChatGPT to create text based on specific instructions, which can serve as augmented data for text classification tasks. Previous studies have approached data augmentation (DA) by either rewriting the existing dataset with ChatGPT or generating entirely new data from scratch. However, it is unclear which method is better without comparing their effectiveness. This study investigates the application of both methods to two datasets: a general-topic dataset (Reuters news data) and a domain-specific dataset (Mitigation dataset). Our findings indicate that: 1. ChatGPT generated new data consistently enhanced model’s classification results for both datasets. 2. Generating new data generally outperforms rewriting existing data, though crafting the prompts carefully is crucial to extract the most valuable information from ChatGPT, particularly for domain-specific data. 3. The augmentation data size affects the effectiveness of DA; however, we observed a plateau after incorporating 10 samples. 4. Combining the rewritten sample with new generated sample can potentially further improve the model’s performance.
97 MATHEMATICS AND COMPUTING↗
HARMONY: Large-Scale Architecture Search for Efficient Hybrid Language Models
As large language models scale to trillions of parameters, their computational and memory requirements present critical challenges for efficient training and deployment. While Mixture of Experts (MoE) architectures enable efficient scaling through sparse parameter activation, and state-space models like Mamba offer linear-time complexity, principled methods for combining these paradigms remain undeveloped. We introduce HARMONY (Hybrid Architecture Research for Mamba, Optimized with Neural efficiencY), a multi-objective evolutionary neural architecture search framework for discovering efficient hybrid language models that integrate Transformer attention mechanisms, Mixture-of-Experts routing, and Mamba state-space components. Through large-scale distributed search using 16,384 MI250X GPUs on the Frontier supercomputer, HARMONY explores a comprehensive design space encompassing six attention variants (MHA, MQA, GQA, MLA, SWA, and Mamba-2), variable MoE configurations with both routed and shared experts, and extensive Mamba hyperparameters. Our framework discovers heterogeneous architectures that balance training performance with computational efficiency through multi-objective optimization incorporating latency penalties and fitness-based selection. Analysis of discovered architectures reveals that optimal hybrid designs favor heterogeneous component mixing rather than homogeneous patterns, with Mamba-2 and Multi-Head Latent Attention (MLA) emerging as preferred mechanisms. Discovered architectures demonstrate superior training efficiency: our best configuration achieves a final perplexity of 1.0874 with 2.38B parameters while processing 4,320 tokens/second, outperforming significantly larger manually designed models. Full-scale evaluation shows HARMONY's top architectures achieve better loss trajectories than equivalently-sized models using state-of-the-art configurations including Mixtral, Jamba, and Samba. Additionally, we demonstrate 91% weak scaling efficiency when training discovered 36B-parameter models across 1,024 GPUs. HARMONY is released as an open framework with comprehensive tools for building and training hybrid models using expert-data-pipeline parallelism, democratizing access to automated architecture design for next-generation language models.
Herron, Emily [ORNL] (ORCID:0000000273008172)↗
An AI-driven framework for evaluating local and state authorities’ permitting processes
The demand for new energy infrastructure is increasing across the United States, but heterogenous permitting processes and embedded requirements across different local jurisdictions can cause project delays, increase “soft costs,” and hinder developer expansion. This study analyzes the variability in local permitting requirements across the U.S. and develops a quantitative approach to describe their clarity and effectiveness in enabling infrastructure project development. By using an Energy Language Model (ELM), a large language model (LLM) for energy technologies, we systematically gathered permitting information from nearly 300 state-, county-, and city-level documents, creating a structured dataset of requirements and procedures on an unprecedented scale and speed. Our analysis revealed that local (city and county) permitting requirement documents are underrepresented compared to state-level guidance documents, which can impede timely and cost-effective installation of new electric infrastructure. Our validation process showed that the final database has an accuracy of approximately 95%. We, further, created a new quantitative method to score permitting requirements for clarity and efficiency, with electric vehicle supply equipment as an initial use case. The average local permitting document scored a 1.8 out of 5, which we interpret as meaning that half of the requirements developers face when installing electric infrastructure are ambiguous, increasing both cost and time. We also created a “Generalized Permit Process”, highlighting common procedural steps and identifying specific opportunities for municipalities to improve their documentation. This research establishes a systematic and scalable framework for evaluating the complexities of local infrastructure permitting processes by combining LLM-powered data collection and quantitative scoring. The framework enables policymakers and developers to identify and mitigate procedural bottlenecks, with the expectation that these improvements can accelerate application review and approval, reduce project costs, and expedite connection to utility distribution grids. As a foundational approach for streamlining local project development processes, this study’s methods are intended to be extended to a wide range of energy applications.
29 ENERGY PLANNING, POLICY, AND ECONOMY↗
ChemReasoner - Catalyst Discovery via Large Language Model-driven Reasoning( 32832-E)
ChemReasoner - Catalyst Discovery via Large Language Model-driven Reasoning
Choudhury, Sutanay [Pacific Northwest National Lab↗
From Rules to Reasoning: A Survey of Large Language Model-Based Approaches to Scientific Hypothesis and Idea Generation
Scientific hypothesis generation represents a fundamental challenge in contemporary research due to exponentially expanding literature volumes and increasing disciplinary specialization. Large language models (LLMs) have emerged as transformative tools for automated scientific discovery, moving beyond traditional rule-based and literature-mining approaches. Four paradigmatic approaches define current LLM-driven hypothesis generation: direct prompting and fine-tuning methods, knowledge-enhanced frameworks integrating retrieval-augmented generation (RAG), multi-agent collaborative systems simulating research teams, and reasoning-focused approaches implementing cognitive architectures. Domain-specific applications demonstrate statistical equivalence to human expert performance in social psychology, experimental validation in biomedical research, and near-expert quality in astronomy. Evaluation methodologies encompass human expert assessment, LLM-as-judge frameworks, and comprehensive benchmarking systems. Technical challenges include hallucination management, knowledge integration limitations, and balancing novelty with feasibility. Future directions emphasize hybrid neural-symbolic architectures and sophisticated human-AI collaboration models for responsible scientific discovery acceleration.
AI-driven discovery↗
Large language model-driven database for thermoelectric materials
Thermoelectric materials have the ability to convert waste heat into electricity, offering a valuable solution for energy harvesting. However, their widespread use is hindered by low conversion efficiency, the reliance on expensive rare earth elements, and the environmental and regulatory concerns associated with lead-based materials. A fast and cost-effective way to identify highly efficient thermoelectric materials is through data-driven methods. These approaches rely on robust and comprehensive datasets to train models. Although there are several databases on thermoelectric materials, there is still a need to collect and integrate experimental data from peer-reviewed research articles to capture diverse compositions and properties of materials. Here, in this work, we developed a comprehensive database of 7,123 thermoelectric compounds, containing key information such as chemical composition, structural detail, seebeck coefficient, electrical and thermal conductivity, power factor, and figure of merit (ZT). We used the GPTArticleExtractor workflow, powered by large language models (LLM), to extract and curate data automatically from the scientific literature published in Elsevier journals. This process enabled the creation of a structured database that addresses the challenges of manual data collection. The open access database could stimulate data-driven research and advance thermoelectric material analysis and discovery.
Database↗
Biological Research and Space Health Enabled by Machine Learning to Support Deep Space Missions
A key science goal of the NASA “Moon to Mars” campaign is to understand how biology responds to the Lunar, Martian, and deep space environments in order to advance fundamental knowledge, reduce risk, and support safe, productive human space missions. Through the powerful emerging approaches of artificial intelligence (AI) and machine learning (ML), a paradigm shift has begun in biomedical science and engineered astronaut health systems, to enable Earth independence and autonomy of mission operations. Here we present an overview of AI/ML architecture to support deep space mission goals, developed with leaders in the field. First, we focus on the fundamental biological research that supports our understanding of physiological responses to spaceflight, and we describe current efforts to support AI/ML research including data standardization and data engineering through maximally open and FAIR (findable, accessible, interoperable, reusable) databases and the generation of AI-ready datasets for reuse and analysis. We also discuss remote data management frameworks for research data as well as environmental and health data that are generated during deep space missions. We highlight several research projects that leverage data standardization and management for fundamental biological discovery to uncover the complex effects of space travel on living systems. Next, we provide an overview of cutting-edge AI/ML approaches that can be integrated to support remote monitoring and analysis during deep space missions, including generative models and large language models to learn the underlying biomedical patterns and predict outcomes or answer questions during off world medical scenarios. We also describe current AI/ML methods to support this research and monitoring through automated cloud-based labs which enable limited human intervention and closed-loop experimentation in remote settings. These labs could support mission autonomy by analyzing environmental data streams, and would be facilitated through in situ analytics capabilities to avoid sending large raw data files through low bandwidth communications. Finally, in the context of deep space missions with limited communications or access to medical advice from Earth, we describe a solution for integrated, real-time mission biomonitoring across hierarchical levels from continuous environmental monitoring, to wearables and point-of-care devices, to molecular and physiological monitoring. We introduce a precision space health system that will ensure that the future of space health is predictive, preventative, participatory and personalized.
artificial intelligence↗
Rapid Adaptation of Chemical Named Entity Recognition Using Few-Shot Learning and LLM Distillation
Named entity recognition (NER) has been widely used in chemical text mining for the automatic identification and extraction of chemical entities. However, existing chemical NER systems primarily focus on scenarios with abundant training data, requiring significant human effort on annotations. This poses challenges for applications in the chemical field, such as catalysis, where many advancements have traditionally relied on trial-and-error investigations and incremental adjustment of variables. This hinders catalysis science and technology progress in addressing emerging energy and environmental crises. In this work, we propose a few-shot NER model that can quickly adapt to extract new types of chemical entities by using only a limited number of annotated examples. Our model employs a metric-learning approach to transfer entity similarity knowledge from high-resource chemical domains (with abundant annotations) to enable effective entity recognition in low-resource specialized domains (limited annotation). We validate the effectiveness of our model on a few-shot chemical NER benchmark built based on six existing chemical NER data sets. Experiments show that the proposed few-shot NER model can achieve reasonable performance with only 5 examples per entity type and shows consistent improvement as the number of examples increases. Furthermore, we demonstrate how the proposed model can be trained with large language model (LLM) annotated data, opening a new pathway for rapid adaptation of NER systems. Furthermore, our approach leverages the knowledge broadness of large language models for chemistry while distilling this knowledge into a lightweight model suitable for efficient and in-house use.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
EV-ELM (Electric Vehicle Policies with the Energy Language Model) [SWR-25-156]
Electric Vehicle Policies with the Energy Language Model (EV-ELM) leverages previous work using Large Language Models (LLMs) to find, download, and parse policy information related to energy infrastructure. In this application, we use LLMs to find policy documents related to the permitting and installation of electric vehicle charging infrastructure. This software contains the code to find, download, and parse these documents, while a related data record in the Open Energy Data Initiative (OEDI) will include the resulting output dataset that can be used for downstream analysis. The EV-ELM repository contains code for the EV-ELM project, which focuses on retrieving and processing EV permitting processes using large language models. The project is composed of two pipelines: (1) a web scraping pipeline for discovering and downloading EV permitting documents, and (2) a document parsing and extraction pipeline that processes the downloaded files to produce structured data. The web scraping pipeline is designed to extract relevant information from various websites, while the document parsing pipeline processes and analyzes the extracted documents to derive meaningful insights. Both pipelines depend on the NLR elm repository, which provides essential tools and functionalities for handling and processing the data. The web scraping pipeline is a modified version of the ordinance_gpt example within the elm repository. It has been adapted to fit the specific requirements of the EV-ELM project, ensuring that it effectively captures and processes the necessary information related to EV permitting.
Olson, Reid [National Laboratory of the Rockies (N↗
Generative AI in Supply Chain Management: Applications, Challenges, and Future Directions
Supply chain management (SCM) is undergoing rapid transformation due to increasing global complexity, demand volatility, and operational disruptions. Generative Artificial Intelligence (GenAI) has emerged as a powerful paradigm capable of synthesizing data, simulating operational scenarios, and enabling adaptive decision-making across supply chain networks. This paper presents a survey of GenAI’s role in SCM, focusing on its applications in predictive analytics, autonomous logistics, and fraud detection. Unlike traditional AI systems that rely primarily on predictive analytics, GenAI models, including large language models, generative adversarial networks, and diffusion-based architectures, enable the creation of synthetic supply chain scenarios and autonomous optimization strategies. This survey provides (1) a taxonomy of GenAI techniques for supply chain applications, (2) a comparative analysis of generative AI approaches with traditional machine learning, reinforcement learning, and blockchain-based methods, and (3) a discussion of key challenges such as data privacy, interpretability, and integration with legacy enterprise systems. Furthermore, we outline open research problems and propose directions for future research toward autonomous, resilient, and sustainable AI-driven supply chains.
15 - GEOTHERMAL ENERGY↗
Artificial Intelligence (AI) Methods for Automating the Impact Tool Evidence Library
INTRODUCTION: The development of the Evidence Library for use with the IMPACT (Informing Mission Planning via Analysis of Complex Tradespaces) probability risk assessment tool involved a multilayered, time intensive process of data collection and analysis by subject matter experts from the Exploration Medical Capability (ExMC) Element Clinical and Science Team to produce clinical findings forms (CliFFs) for 120 medical conditions. Artificial Intelligence Large Language Models (LLMs) can be leveraged to facilitate this process, thus reducing labor and time. TOPIC: CliFFs contain information about medical conditions as they pertain to spaceflight. This includes condition definitions, incidence data, crew task impairment estimates caused by conditions, treatment protocols and references to literature used for gathering condition evidence. Guided by the Evidence Library Methods document and the CliFF development instructions, a team has leveraged Microsoft Azure AI services and open-source documentation to construct an AI-assisted automated pipeline for CliFF development. This process is designed to search, retrieve, and evaluate the applicable data, and ultimately generate a completed CliFF. The LLM evaluates the relevance of each of the source materials to spaceflight, either as direct evidence or as an analog. The model extracts keywords and generates brief summaries to enhance search and retrieval in later stages of CliFF development. For instance, it can calculate epidemiological statistical data, such as incidence rates and the likelihood of best or worst-case scenarios. APPLICATION: Large Language Models (LLMs) can efficiently summarize large amounts of text. Leveraging this technology will automate data retrieval and evidence gathering for medical databases, like the IMPACT tool, by aiding in the labor-intensive process of analyzing large bodies of literature and organizing it into a formatted document like a CliFF. This added efficiency will enable expeditious expansion of the Evidence Library with additional medical conditions and update previous CLiFFs as new technology becomes available.
Ali Al↗
From Research to Reality -- Challenges and Opportunities in Complex System Design
The scope and scale of our interconnected society requires that we view the world through a complex system lens, where numerous parts interact, and emergent behaviors are the norm. Understanding complex systems is critical for decision-making and policy development in domains such as ecological systems, financial markets, supply chains, and global transportation systems, where decision-makers need reliable information to predict the impact of decisions that may play out over decades. Traditional approaches to the research that produces this information are often insufficient, where hypotheses are tested in an isolated environment, and where the results may not carry over to the integrated system. There are a multitude of advancements in system engineering, artificial intelligence, and test and evaluation that are emerging to meet the challenge. Approaches such as agile development, model-based system engineering, design of experiments, large language models, and formal ontologies are providing ways to manage complexity and increase our collective ability to make the changes that we want to see in the world. In this talk I will provide a few examples related to the architecture of the National Airspace System (NAS), where we have investigated the use of large language models, basic formal ontology, and applied category theory to help researchers and system engineers be more effective in this complex design space. Bio: Dr. Ian Levitt’s current research focus is on the complex evolution of the National Airspace System. Prior to joining NASA in 2020, he was with the FAA leading international standards and national laboratory development for the agency. Dr. Levitt earned his PhD in mathematics from Rutgers University in 2009. His mission is to promote a healthy and continuous transformation of society through open information and cooperation.
Ian Levitt↗
Artificial Intelligence (AI) Methods for Augmenting the IMPACT Tool Evidence Library
Development of the Evidence Library for use with the IMPACT probability risk assessment tool took several years and involved a staggering amount of effort from a multi-disciplinary team. A very significant amount of the labor effort to collect, assess and finalize the Clinical Finding Form (CliFF) for each of the 119 medical conditions was provided by physician subject matter experts from the Exploration Medical Capability (ExMC) Element Clinical and Science Team. Many AI tools such as ChatGPT are excellent at summarizing large amounts of information and the current project was initiated to determine how such tools might streamline laborious processes, e.g., review and summarization of many scientific research publications, to execute key steps more efficiently in the process of developing CliFFs. The process for collecting the evidence which is found in the CliFFs is well documented in the Evidence Library Methods document (ELM; HRP-48036*). Using ELM and the CliFF development instructions as a guideline, a team of developers is leveraging Microsoft Azure AI tools and services along with open-source frameworks, to construct an AI-assisted automated pipeline. This pipeline is designed to search, retrieve, and process the necessary data sources, and ultimately help generate the final version of a CliFF. Currently, the large language model evaluates the relevance of each source material to spaceflights, either as direct evidence or as an analog. Additionally, the model assists in extracting keywords and generating brief summaries to enhance augmented retrieval and search processes in later stages of CliFF development. Once the data is ready, the model can perform semantic search and retrieval, generating and extracting valuable information for the CliFF. For instance, it can handle epidemiological statistical data, such as incidence rates and the likelihood of best or worst-case scenarios. The steps that required reading and summarizing articles were viewed as providing the greatest return on investment since large language models are very efficient and accurate in summarizing large amounts of text. Since labor effort to complete the original CliFF was not recorded with sufficient granularity, comparisons with an AI tool-generated CliFF will provide merely an approximation of time saved. Upon completion of the process, the CliFF for the medical condition “appendicitis” generated with the support of AI-based methods will serve as a proof-of-concept and will be compared to the original appendicitis CliFF to determine if use of the tools resulted in content and conclusory similarity. Based upon the results from face validation of the two CliFFs, modifications to the process will be made if necessary and additional condition CliFFs will be evaluated. Ultimately, CliFFs for the entire set of medical conditions will be created with the assistance of AI tools. Depending on the cost savings realized, CliFFs for additional medical conditions can be created to expand the Evidence Library. Future direction includes specifying the characteristics of the reviewer (prompting the AI tools to generate output assuming the reviewer is a sub-specialist physician, or nurse or EMT/medic) to determine if the effects on AI-generated output are different based on knowledge, skills and abilities. *Exploration Medical Capability Evidence Library Methods, HRP-48036 Rev A, July 2022.
Ali Al↗
Artificial Intelligence Medical Support for Long-Duration Space Missions
We envision an artificial intelligence (AI) based system that will provide support and recommendations to the crew medical officer (CMO) and ground flight surgeon during long-duration space missions. Such a system would be pretrained on the knowledgebase of clinical knowledge on Earth, minimizing the amount of Earth data that needs to be transferred into space. Then during deployment, the system would be constantly refined through active learning from diverse streams of data from sensors in the spacecraft, data collected daily from individual astronauts, and human-in-the-loop feedback from the crew. The model could be interrogated for predictions and recommendations on personalized crew health based on the overall status of the spacecraft, medicinal stores, and status of other crew members. Adaptation techniques would be used to incorporate spaceflight data that have very different distributions from the training data due to the extreme environment. Edge computing and the most advanced neuromorphic processing would enable computation in scenarios with low power and bandwidth, while dimensionality reduction would be employed to ensure that the input data streams from spaceflight are as small as possible. In order to realize this long-term vision, several hardware and software aspects need to be developed and assembled. First, models pretrained on Earth biomedical data would need to be evaluated for predictive accuracy, and the best one selected. That model would need to be adapted to learn from diverse, sparse, and inconsistently measured data streams, as well as human-in-the-loop feedback. A data integration, standardization, and dimensionality reduction methodology would need to be developed to handle all data types and feed them into the model. Once the software and data infrastructure is developed, it would need to be integrated with small footprint compute processors and tested in high-radiation, high-vibration, unregulated temperature situations. As a short-term goal, we recommend to focus on the development of the data and model software structure. Several large language models (LLM) already exist that have been trained on Earth biomedical and clinical knowledgebases, including BioMedLLM, Med-PaLM, SPOKE LLM, and Foresight. These models need to be evaluated for accuracy and the best one chosen for a proof-of-concept structure, while maintaining awareness of the accelerating AI field and incorporating any newly improved model architectures as needed. Then, we recommend to develop a database of synthetic data types to mimic the diverse data streams that are expected in a long-duration space mission. This should include environmental and microbial data from the spacecraft, non-invasive data from wearables and point-of-care devices employed by astronauts, and more invasive molecular and physiological monitoring of clinical and biomarker data from astronauts. The data standardization methodology should be developed, and these data streams used to refine the clinical LLM. Several scenarios should be developed that could plausibly come up in a long-duration space mission, and changes or aberrations introduced to the data at specific times to mimic these scenarios. Then, question and answer tasks should be designed to interrogate the model for predictions and recommendations, with acceptable answers already identified.
Artificial Intelligence↗
FM4NPP: A Scaling Foundation Model for Nuclear and Particle Physics
Large language models have revolutionized artificial intelligence by enabling large, generalizable models trained through self-supervision. This paradigm has inspired the development of scientific foundation models (FMs). However, applying this capability to experimental particle physics is challenging due to the sparse, spatially distributed nature of detector data, which differs dramatically from natural language. This work addresses if an FM for particle physics can scale and generalize across diverse tasks. We introduce a new dataset with more than 11 million particle collision events and a suite of downstream tasks and labeled data for evaluation. We propose a novel self-supervised training method for detector data and demonstrate its neural scalability with models that feature up to 188 million parameters. With frozen weights and task-specific adapters, this FM consistently outperforms baseline models across all downstream tasks. The performance also exhibits robust data-efficient adaptation. Further analysis reveals that the representations extracted by the FM are task-agnostic but can be specialized via a single linear mapping for different downstream tasks.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
MoE-Inference-Bench: Performance Evaluation of Mixture of Expert Large Language and Vision Models
Mixture of Experts (MoE) models have enabled the scaling of Large Language Models (LLMs) and Vision Language Models (VLMs) by achieving massive parameter counts while maintaining computational efficiency. However, MoEs introduce several inference-time challenges, including load imbalance across experts and the additional routing computational overhead. To address these challenges and fully harness the benefits of MoE, a systematic evaluation of hardware acceleration techniques is essential. We present MoE-Inference-Bench, a comprehensive study to evaluate MoE performance across diverse scenarios. We analyze the impact of batch size, sequence length, and critical MoE hyperparameters such as FFN dimensions and number of experts on throughput. We evaluate several optimization techniques on Nvidia H100 GPUs, including pruning, Fused MoE operations, speculative decoding, quantization, and various parallelization strategies. Our evaluation includes MoEs from the Mixtral, DeepSeek, OLMoE and Qwen families. The results reveal performance differences across configurations and provide insights for the efficient deployment of MoEs.
Chitty-Venkata, Krishna Teja↗