Search NASA⌕ Search

SEARCH · Search NASA

Results for “Language model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25

Scientific Open-Source Software Is Less Likely to Become Abandoned Than One Might Think! Lessons from Curating a Catalog of Maintained Scientific Software

Scientific software is essential to scientific innovation and in many ways it is distinct from other types of software. Abandoned (or unmaintained), buggy, and hard to use software, a perception often associated with scientific software can hinder scientific progress, yet, in contrast to other types of software, its longevity is poorly understood. Existing data curation efforts are fragmented by science domain and/or are small in scale and lack key attributes. We use large language models to classify public software repositories in World of Code into distinct scientific domains and layers of the software stack, curating a large and diverse collection of over 18,000 scientific software projects. Using this data, we estimate survival models to understand how the domain, infrastructural layer, and other attributes of scientific software affect its longevity. We further obtain a matched sample of non-scientific software repositories and investigate the differences. We find that infrastructural layers, downstream dependencies, mentions of publications, and participants from government are associated with a longer lifespan, while newer projects with participants from academia had shorter lifespan. Against common expectations, scientific projects have a longer lifetime than matched non-scientific open-source software projects. We expect our curated attribute-rich collection to support future research on scientific software and provide insights that may help extend longevity of both scientific and other projects.

Malviya Thakur, Addi [ORNL] (ORCID:000000022681999↗

Enhancing ChatPORT with CUDA-to-SYCL Kernel Translation Capability

Large Language Models (LLMs) have shown strong capabilities in general code translation. However, code translation involving parallel programming models remains largely unexplored. This work enhances the capabilities of code LLMs in CUDA-to-SYCL kernel translation with parameter-efficient fine-tuning. The resultant fine-tuned LLM, called ChatPORT, is an effort to provide high-fidelity translations from one programming model to another. We describe the preparation of datasets from heterogeneous computing benchmarks for model fine-tuning and testing, the parameter-efficient fine-tuning of 19 open-source code models ranging in size from 0.5 to 34 billion parameters and evaluate the correctness rates of the SYCL kernels by the fine-tuned models. The experimental results show that most code models fail to translate CUDA codes to SYCL correctly. However, fine-tuning these models using a small set of CUDA and SYCL kernels can enhance the capabilities of these models in kernel translation. Depending on the sizes of the models, the correctness rate ranges from 19.9% to 81.7% for a test dataset of 62 CUDA kernels.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗

CAFE AU LAIT: Compute-Aware Federated Augmented Low-Rank AI Training

Federated finetuning is crucial for unlocking the knowledge embedded in pretrained Large Language Models (LLMs) when data are geographically distributed across clients. Unlike finetuning with data from a single institution, federated finetuning allows collaboration across multiple institutions, enabling the utilization of diverse and decentralized datasets while preserving data privacy. Given the high computing costs of LLM training and the emphasis on energy efficiency in Federated Learning (FL), Low-Rank Adaptation (LoRA) has emerged as a widely adopted algorithm due to its significantly reduced number of trainable parameters. However, this assumes that all data silos have the necessary computing resources to compute local updates of LLMs. Nevertheless, in practice, the computing resources across clients are highly heterogeneous: while some may have access to hundreds of GPUs, others might have limited or no GPU access. Recently, federated finetuning using synthetic data has been proposed, allowing clients to participate in a collaborative training run without training LLMs locally. However, our experimental results reveal a performance gap between models trained using synthetic data and those trained using local updates. Motivated by the observed heterogeneity in computing resources and the performance gap, we propose a novel two-stage algorithm that leverages the storage and computing capabilities of a strong server. In the first stage, under the coordination of the strong server, clients with limited computing resources collaborate to generate synthetic data, which is transferred to and stored on the strong server. In the second stage, the strong server uses this synthetic data on behalf of the resource-constrained clients to perform federated LoRA finetuning alongside clients with sufficient computing resources. This approach ensures that all clients can participate in the finetuning process. Experimental results demonstrate that incorporating local updates from even a small fraction of clients improves performance compared to using synthetic data for all clients. Furthermore, we incorporate the Gaussian mechanism in both stages to guarantee client-level differential privacy.

Wang, Jiayi [ORNL]↗

ChatMPI: LLM-Driven MPI Code Generation for HPC Workloads

The Message Passing Interface (MPI) standard plays a crucial role in enabling scientific applications for parallel computing and is an essential component in high-performance computing (HPC). However, implementing MPI code manually—especially applying a proper domain decomposition and communication pattern—is a challenging and error-prone task. We present ChatMPI, an AI assistant for MPI parallelization of sequential C codes. In our analysis, we focus on testing six essential HPC workloads, which are based on Basic Linear Algebra Subprograms levels 1, 2, and 3 as well as sparse, stencil, and iterative operations. We analyze the process of creating ChatMPI by using the ChatHPC library. This lightweight large language model (LLM)–based infrastructure enables HPC experts to efficiently create and supervise trustworthy AI capabilities for critical HPC software tasks. We study the data required for training (fine-tuning) ChatMPI to generate parallel codes that not only use MPI syntax correctly but also apply HPC techniques to reduce memory communication and maximize performance by using proper work decomposition. With a relatively small training dataset composed of a few dozen prompts and fewer than 15 minutes of fine-tuning on one node equipped with two NVIDIA H100 GPUs, ChatMPI elevates trustworthiness for MPI code generation of current LLMs (e.g., Code Llama, ChatGPT-4o and ChatGPT 5). Additionally, we evaluate the performance of the MPI codes generated by ChatMPI in comparison with the ones generated by ChatGPT-4o and ChatGPT-5. The codes generated by ChatMPI provide up to a 4 × boost in performance by using better problem decomposition, communication patterns, and HPC techniques (e.g., communication avoiding).

Valero Lara, Pedro [ORNL] (ORCID:0000000214794310)↗

HERON

HERON is a modeling toolset and plugin for RAVEN to accelerate stochastic technoeconomic assessment of the economic viability of various grid-energy system configurations, especially with application to electrical grids and integrated energy systems (IES). The frameworks enabling this toolset are INL's RAVEN and its TEAL plugin. The toolset includes automated RAVEN workflow generation tools for performing stochastic analysis and optimization driven by technoeconomic factors. The workflows generated employ RAVEN for parametric or optimizing study of grid component characteristics, and again for stochastic sampling of regulated and deregulated markets, load, and weather histories. The toolset further provides methods for economically-optimal dispatch of system components given the stochastic samples of market, load, and weather histories. The toolset allows assembling of existing and new models through the RAVEN interface to the Modelica process modeling language.

Talbot, PaulW↗

Transformer eXplainability and eXploration

The Transformer eXplainability and eXploration library is intended to aid in the explorability and explainability of transformer classification networks, or transformer language models with sequence classification heads. The basic function of this library is to take a trained transformer and test/train dataset and produce an ipywidget dashboard which can be displayed in a jupyter notebook or in jupyter lab.

Martindale, Nathan [Oak Ridge National Lab. (ORNL)↗

Spawn [SWR-SWR-20-10]

Spawn is a next generation building simulation engine designed to build on the strengths of NREL's venerable EnergyPlus engine, and add new advanced HVAC and controls modeling capability by incorporating the Modelica equation based modeling language. In effect, Spawn is a hybrid of the existing EnergyPlus and Modelica software products, with some new software to provide glue between the components in order to provide a seamless end user experience.

Li, Yanfei↗

MFDn Kernels v1.0

MFDn kernels is a collection of small subroutines designed to capture essential characteristics of the Many-Fermion Dynamics---nuclear (MFDn) application (DOI: 10.1016/j.procs.2010.04.012). It features multiple implementations with different programming models/languages of simplified routines analogous to those found in the full application. The main use is to explore the performance and utility of different programming models, languages and compilers.

Cook, Brandon↗

CACTUS

CACTUS is an innovative LLM‐based agent that combines the cognitive capabilities of large language models with domain‐specific tools. By integrating cheminformatics tools, CACTUS assists researchers in tasks such as molecular property prediction, similarity searching, and drug‐likeness assessment.

Kumar, Neeraj↗

CLIP Representation Enhanced Predicate Estimation

This software provides functions to perform visual predicate estimation by leveraging the pre-trained vision-language model CLIP. In addition to training the prediction estimator with a labeled dataset, it also performs performance evaluation and visualization of results.

Jayaraman Thiagarajan, Jayaraman↗

Trajectory Balance with Asynchrony

Finetune large language models (LLMs) quicker via parallelization. Specifically, this implements asynchronous reinforcement learning with a trajectory balance objective function

Bartoldson, BrianR [Lawrence Livermore National La↗

Prompt-Based Development of Domain-Specific Taxonomies

Framework for interacting with prompt-based LLMs (Large Language Models such as, but not limited to, ChatGPT) in order to develop domain specific taxonomies. The software itself is a series of prompts designed to extract hierarchical taxonomy categories from an LLM, as well as guidelines for how to fine-tune LLMs in order to provide better domain-specific taxonomic categories.

Grundy, Jon [Pacific Northwest National Laboratory↗

tether

Tether is a python module for benchmarking and assessing large language model (LLMs) performance at generic scientific tasks. The code generates benchmarks, uses the benchmark to prompt LLMs through automatic programming interfaces (APIs), and then logs the number of prompts an LLM correctly answers and presents the results as a completed benchmark.

Kaiser, Bryan [Los Alamos National Laboratory]↗

Open AI Co-Scientist - Hypothesis Evolution System

An AI-powered system for generating, reviewing, ranking, and evolving research hypotheses. It leverages Large Language Models (LLMs) for various tasks, including hypothesis generation, reflection, and comparison. The system is designed to assist researchers in exploring a research space and identifying promising hypotheses.

Liao, Chunhua↗

Activation Domain Hunter (ADhunter) v2.0

ADhunter is a software program that enables accurate identification and quantification of transcriptional activation domains. Unlike previous software, ADhunter uses protein representations from a pre-trained protein language model, model ensembling, and a training dataset from a diverse sampling of protein sequence space for state-of-the-art performance. These advantages enable improved perception of transcriptional activation domains across sequence space that can be used for mapping natural genetic circuits and engineering synthetic genetic circuits. In particular, ADhunter enables fine-tuned control of gene expression through synthetic transcription factors that can be used for complex control of cellular programs.

Waldburger, Lucas [Lawrence Berkeley National Labo↗

ellora-spack-gen

The project contains software to analyze the behavior of large language models at code generation tasks. Publicly available information is used to generate Spack package recipes for HPC developers. The software contains orchestration tooling, analysis, and plotting functionality.

Melone, CaetanoN↗