SEARCH · Search NASA

Results for “Small Language Model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

133 records · Page 8

All sources NASA NTRS DOE OSTI

DOE OSTI2025

CAFE AU LAIT: Compute-Aware Federated Augmented Low-Rank AI Training

Federated finetuning is crucial for unlocking the knowledge embedded in pretrained Large Language Models (LLMs) when data are geographically distributed across clients. Unlike finetuning with data from a single institution, federated finetuning allows collaboration across multiple institutions, enabling the utilization of diverse and decentralized datasets while preserving data privacy. Given the high computing costs of LLM training and the emphasis on energy efficiency in Federated Learning (FL), Low-Rank Adaptation (LoRA) has emerged as a widely adopted algorithm due to its significantly reduced number of trainable parameters. However, this assumes that all data silos have the necessary computing resources to compute local updates of LLMs. Nevertheless, in practice, the computing resources across clients are highly heterogeneous: while some may have access to hundreds of GPUs, others might have limited or no GPU access. Recently, federated finetuning using synthetic data has been proposed, allowing clients to participate in a collaborative training run without training LLMs locally. However, our experimental results reveal a performance gap between models trained using synthetic data and those trained using local updates. Motivated by the observed heterogeneity in computing resources and the performance gap, we propose a novel two-stage algorithm that leverages the storage and computing capabilities of a strong server. In the first stage, under the coordination of the strong server, clients with limited computing resources collaborate to generate synthetic data, which is transferred to and stored on the strong server. In the second stage, the strong server uses this synthetic data on behalf of the resource-constrained clients to perform federated LoRA finetuning alongside clients with sufficient computing resources. This approach ensures that all clients can participate in the finetuning process. Experimental results demonstrate that incorporating local updates from even a small fraction of clients improves performance compared to using synthetic data for all clients. Furthermore, we incorporate the Gaussian mechanism in both stages to guarantee client-level differential privacy.

Wang, Jiayi [ORNL]↗

NASA NTRS2004

Wave Propagation in 2-D Granular Matrix and Dust Mitigation of Fabrics for Space Exploration Mission

Wave Propagation study is essential to exploring the soil on Mars or Moon and Dust Mitigation is a necessity in terms of crew's health in exploration missions. The study of Dust Mitigation has a significant impact on the crew s health when astronauts track dust back into their living space after exploration trips. We are trying to use piezoelectric fiber to create waves and vibrations at certain critical frequencies and amplitudes so that we can shake the particles off from the astronaut s fabrics. By shaking off the dust and removing it, the astronauts no longer have to worry about breathing in small and possibly hazardous materials, when they are back in their living quarters. The Wave Propagation in 2-D Granular Matrix studies how the individual particles interact with each other when a pressure wave travels through the matrix. This experiment allows us to understand how wave propagates through soils and other materials. By knowing the details about the interactions of particles when they act as a medium for waves, we can better understand how wave propagates through soils and other materials. With this experiment, we can study how less gravity effects the wave propagation and hence device a way to study soils in space and on Moon or Mars. Some scientists treat the medium that waves travel through as a "black box", they did not pay much attention to how individual particles act as wave travels through them. With this data, I believe that we can use it to model ways to measure the properties of different materials such as density and composition. In order to study how the particles interact with each other, I have continued Juan Agui's experiment of the effects of impacts on a 2-D matrix. By controlling the inputs and measuring the outputs of the system, I will be able to study now the particles in that system interact with each other. I will also try to model this with the software called PFC2D in order to obtain theoretical data to compare with the experiment. PFC2D is a program that allows the user to control the number of particle's characteristic, and the environment of the particle. With this I can run simulations that mimic the impulse test. This software uses a language called FISH, probably created by the creator of the software. This means that in order to model anything, one must use the command terminal instead of GUI's. I will also use this program to simulate the Moon/Mars simulate adhering to the fabric for the Dust Mitigation project. My goals for this summer are just to complete preliminary studies of the feasibility of the Shaking Fabric, learn the PFC-2D program, and to complete building and testing the wave propagation experiment.

Thanh, Phi Hung X.↗

NASA NTRS1999

Autonomous Command Operations of the WIRE Spacecraft

This paper presents operational innovations which will be introduced on NASA's Wide Field Infrared Explorer (WIRE) mission. These innovations include an end-to-end design architecture for an autonomous commanding capability for the uplink of command loads during unattended station contacts. The WIRE mission is the fifth and final mission of NASA's Goddard Space Flight Center Small Explorer (SMEX) series to be launched in March of 1999. Its primary mission is the targeting of deep space fields using an ultra-cooled infrared telescope. Due to its mission design WIRE command loads are large (approximately 40 Kbytes per 24 hours) and must be performed daily. To reduce the cost of mission operations support that would be required in order to uplink command loads, the WIRE Flight Operations Team has implemented all autonomous command loading capability. This capability allows completely unattended operations over a typical two-day weekend period. The key factors driving design and implementation of this capability were: 1) integration with already existing ground system autonomous capabilities and systems, 2) the desire to evolve autonomous operations capabilities based upon previous SMEX operations experience - specifically the TRACE mission, 3) integration with ground station operations - both autonomous and man-tended, 4) low cost and quick implementation, and 5) end-to-end system robustness. A trade-off study was performed to examine these factors in light of the low-cost, higher-risk SMEX mission philosophy. The study concluded that a STOL (Spacecraft Test and Operations Language) based script, highly integrated with other scripts used to perform autonomous operations, was best suited given the budget and goals of the mission. Each of these factors is discussed in addition to use of the TRACE mission as a testbed for autonomous commanding prior to implementation on WIRE. The capabilities implemented on the WIRE mission are an example of a low-cost, robust, and efficient method for autonomous command loading when implemented with other autonomous features of the ground system. They call be used as a design and implementation template by other missions interested in evolving toward autonomous and lower cost operations. Additionally, the WIRE spacecraft will be used as an operational testbed upon completion of its nominal mission later in 1999. One idea being studied is advanced on-board modeling. Advanced on-board modeling techniques will be used to more efficiently display the spacecraft state. This health and safety information could be used by engineers on the ground or could be used by tile spacecraft for its own assessments. Additionally, this same state information could also be input into the event-driven scheduling system, as the scheduling system will need to assess the spacecraft state before undertaking a new activity. Advanced modeling techniques are being evaluated for a number of NASA missions including The Next Generation Space Telescope (NGST), which is scheduled to launch in 2007.

Walyus, Keith↗

NASA NTRS1997

Cooperative Data Sharing: Simple Support for Clusters of SMP Nodes

Libraries like PVM and MPI send typed messages to allow for heterogeneous cluster computing. Lower-level libraries, such as GAM, provide more efficient access to communication by removing the need to copy messages between the interface and user space in some cases. still lower-level interfaces, such as UNET, get right down to the hardware level to provide maximum performance. However, these are all still interfaces for passing messages from one process to another, and have limited utility in a shared-memory environment, due primarily to the fact that message passing is just another term for copying. This drawback is made more pertinent by today's hybrid architectures (e.g. clusters of SMPs), where it is difficult to know beforehand whether two communicating processes will share memory. As a result, even portable language tools (like HPF compilers) must either map all interprocess communication, into message passing with the accompanying performance degradation in shared memory environments, or they must check each communication at run-time and implement the shared-memory case separately for efficiency. Cooperative Data Sharing (CDS) is a single user-level API which abstracts all communication between processes into the sharing and access coordination of memory regions, in a model which might be described as "distributed shared messages" or "large-grain distributed shared memory". As a result, the user programs to a simple latency-tolerant abstract communication specification which can be mapped efficiently to either a shared-memory or message-passing based run-time system, depending upon the available architecture. Unlike some distributed shared memory interfaces, the user still has complete control over the assignment of data to processors, the forwarding of data to its next likely destination, and the queuing of data until it is needed, so even the relatively high latency present in clusters can be accomodated. CDS does not require special use of an MMU, which can add overhead to some DSM systems, and does not require an SPMD programming model. unlike some message-passing interfaces, CDS allows the user to implement efficient demand-driven applications where processes must "fight" over data, and does not perform copying if processes share memory and do not attempt concurrent writes. CDS also supports heterogeneous computing, dynamic process creation, handlers, and a very simple thread-arbitration mechanism. Additional support for array subsections is currently being considered. The CDS1 API, which forms the kernel of CDS, is built primarily upon only 2 communication primitives, one process initiation primitive, and some data translation (and marshalling) routines, memory allocation routines, and priority control routines. The entire current collection of 28 routines provides enough functionality to implement most (or all) of MPI 1 and 2, which has a much larger interface consisting of hundreds of routines. still, the API is small enough to consider integrating into standard os interfaces for handling inter-process communication in a network-independent way. This approach would also help to solve many of the problems plaguing other higher-level standards such as MPI and PVM which must, in some cases, "play OS" to adequately address progress and process control issues. The CDS2 API, a higher level of interface roughly equivalent in functionality to MPI and to be built entirely upon CDS1, is still being designed. It is intended to add support for the equivalent of communicators, reduction and other collective operations, process topologies, additional support for process creation, and some automatic memory management. CDS2 will not exactly match MPI, because the copy-free semantics of communication from CDS1 will be supported. CDS2 application programs will be free to carefully also use CDS1. CDS1 has been implemented on networks of workstations running unmodified Unix-based operating systems, using UDP/IP and vendor-supplied high- performance locks. Although its inter-node performance is currently unimpressive due to rudimentary implementation technique, it even now outperforms highly-optimized MPI implementation on intra-node communication due to its support for non-copy communication. The similarity of the CDS1 architecture to that of other projects such as UNET and TRAP suggests that the inter-node performance can be increased significantly to surpass MPI or PVM, and it may be possible to migrate some of its functionality to communication controllers.

DiNucci, David C.↗

NASA NTRS2025

A Preliminary Study on the Feasibility of Large Language Models for Detecting Micro-Behaviors Among Team Members in Space Missions

Large-language models (LLMs) have been recently used for spoken language understanding (SLU) to infer meaning and semantics from speech in tasks such as speaker intent and sentiment classification. Due to being trained on large amounts of data, and their ability to understand context and relationships between words, LLMs are competent, enabling them to generalize across tasks without requiring many task-specific training samples. This research examines the feasibility of few-shot learning in LLMs for detecting subtle, brief, and possibly unconscious interactions between team members, called ``micro-behaviors," and provides insights into the appropriate design of LLMs for this task. Our data came from 5 teams participating in a 45-day mission at the US National Aeronautics and Space Administration’s (NASA) Human Exploration Research Analog (HERA). More specifically we used data collected from team interaction battery (TIB) tasks teams performed five times in-mission which comprise an average 1.5 hours of conversation data per day. Micro-behaviors were coded according to an adapted version of Smith & Griffins (2022) theoretical framework in terms of Violation (i.e., presence of valenced behavior, uplifting/positive or discouraging/negative), Intensity (i.e., force of behavior in terms of how uplifting or discouraging is the behavior), and Intent (i.e., motive of the behavior in terms of whether it was deliberate or unintentional). We explore the ability of LLMs to detect the presence and intensity of micro-behaviors. We examine employing and fine-tuning readily available LLMs (i.e., RoBERTa, DistilBERT), as well as prompting state-of-the-art sequence classification models (i.e., Llama-2, Llama-3). In a total of 13,058 conversational turns (17.8% uplifting, 3.3% discouraging, 75.76% neutral, 3.14% nulls), we compute the macro F1-score of the 3-way micro-behavior classification task (i.e., classifying among uplifting, discouraging, and neutral; 33% chance). Results indicate that the RoBERTa model achieves a F1-score of 36.2% (uplift: 43.3% precision (P), 15.1% recall (R); discourage: 20% P, 0.5% R). These results significantly improve when we augment the data via paraphrasing in the RoBERTa model, reaching a 41.2% macro F1-score (uplift: 37.7% P, 86.3% R; discourage: 3.5% P, 1.8% R). Finally, the Llama-2 model with 3-shot prompting yields 38% macro F1-score (uplift: 28.7% P, 20% R; discourage: 7.2% P, 18% R), which is slightly better compared to the RoBERTa model without data augmentation, highlighting the effectiveness of sequence classification models in detecting minority classes with a small sample size. Findings indicate that LLMs hold potential to detect subtle behaviors in conversations, which could be valuable in assessing team behavior in space exploration missions. Future studies will evaluate the performance of different LLM prompting strategies or fine-tuning methods.

Ankush Raut↗

NASA NTRS1994

Understanding software faults and their role in software reliability modeling

This study is a direct result of an on-going project to model the reliability of a large real-time control avionics system. In previous modeling efforts with this system, hardware reliability models were applied in modeling the reliability behavior of this system. In an attempt to enhance the performance of the adapted reliability models, certain software attributes were introduced in these models to control for differences between programs and also sequential executions of the same program. As the basic nature of the software attributes that affect software reliability become better understood in the modeling process, this information begins to have important implications on the software development process. A significant problem arises when raw attribute measures are to be used in statistical models as predictors, for example, of measures of software quality. This is because many of the metrics are highly correlated. Consider the two attributes: lines of code, LOC, and number of program statements, Stmts. In this case, it is quite obvious that a program with a high value of LOC probably will also have a relatively high value of Stmts. In the case of low level languages, such as assembly language programs, there might be a one-to-one relationship between the statement count and the lines of code. When there is a complete absence of linear relationship among the metrics, they are said to be orthogonal or uncorrelated. Usually the lack of orthogonality is not serious enough to affect a statistical analysis. However, for the purposes of some statistical analysis such as multiple regression, the software metrics are so strongly interrelated that the regression results may be ambiguous and possibly even misleading. Typically, it is difficult to estimate the unique effects of individual software metrics in the regression equation. The estimated values of the coefficients are very sensitive to slight changes in the data and to the addition or deletion of variables in the regression equation. Since most of the existing metrics have common elements and are linear combinations of these common elements, it seems reasonable to investigate the structure of the underlying common factors or components that make up the raw metrics. The technique we have chosen to use to explore this structure is a procedure called principal components analysis. Principal components analysis is a decomposition technique that may be used to detect and analyze collinearity in software metrics. When confronted with a large number of metrics measuring a single construct, it may be desirable to represent the set by some smaller number of variables that convey all, or most, of the information in the original set. Principal components are linear transformations of a set of random variables that summarize the information contained in the variables. The transformations are chosen so that the first component accounts for the maximal amount of variation of the measures of any possible linear transform; the second component accounts for the maximal amount of residual variation; and so on. The principal components are constructed so that they represent transformed scores on dimensions that are orthogonal. Through the use of principal components analysis, it is possible to have a set of highly related software attributes mapped into a small number of uncorrelated attribute domains. This definitively solves the problem of multi-collinearity in subsequent regression analysis. There are many software metrics in the literature, but principal component analysis reveals that there are few distinct sources of variation, i.e. dimensions, in this set of metrics. It would appear perfectly reasonable to characterize the measurable attributes of a program with a simple function of a small number of orthogonal metrics each of which represents a distinct software attribute domain.

Munson, John C.↗

DOE OSTI2021

Data from: "Warming and provenance limit tree recruitment across and beyond the elevation range of subalpine forest"

This data package contains data used to support conclusions drawn in “Warming and provenance limit tree recruitment across and beyond the elevation range of subalpine forest”, by Kueppers et al. 2017. Data were collected in field sites within the Alpine Treeline Warming Experiment (ATWE), located on Niwot Ridge, on the eastern slope of the Colorado Rocky Mountains, USA. Files containing geospatial data are also included, to provide additional locational context.There are four document formats associated with this archive: three comma-separated values (.csv) files, three Microsoft Excel (.xlsx) files, one .pdf data user’s guide, four keyhole markup language (.kml) files, and a compressed folder containing seven ESRI shapefiles (.shp). The .csv files can be opened using any simple text-editor software, R, or Microsoft Excel. The .xlsx files can only be opened using Microsoft Excel. The .kml file can be opened by Google Earth and Google Maps, and the shapefiles can be opened with any GIS application compatible with the file type, such as ESRI’s ArcGIS, and QGIS.We provide two versions of the seedling data file: “PIEN_PIFLseedlings20150522_20150525rev12222020.csv/.xlsx” (hereafter PIEN_PIFLseedlings2015) and “PIEN_PIFLseedlings20160408rev12222020.csv/.xlsx” (hereafter PIEN_PIFLseedlings2016). PIEN_PIFLseedlings2015 contains the data we used in the paper. PIEN_PIFLseedlings2016 contains an updated version of these data that includes sampling from later years. The main differences between the two files lie in the columns titled “k[YEAR],” which describe the number of seedlings that were killed in a particular year. In PIEN_PIFLseedlings2016, there also is an additional year of data for k2015, and k2014 also has additional data input for the 2014 cohort. Additionally, in years 2010-2014, there are minor differences in the number of seedlings killed -- in as few as 0 plots (in 2011) to as many as 5 plots (in 2014) -- due to errors in data input that were rectified in later years.------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Upslope range shifts by subalpine tree species are a widely anticipated effect of climate change. Climate niche models predict subalpine forests to expand upslope, given more suitable growing conditions for adult trees. However, these models do not take into account climates required for successful seedling recruitment and establishment, an essential element for expansion. Further, localized upper treeline populations are hypothesized to contain favorable traits for colonizing the alpine. To test these expectations and to expand our knowledge of seedling recruitment under climate change, we designed a common garden, climate-warming experiment spread across an elevation gradient at Niwot Ridge in the Colorado Rocky Mountains. We focus on two widespread Western North American species, Engelmann spruce (Picea engelmannii Parry ex. Engelm) and limber pine (Pinus flexilis James), which occur at treeline. While the former is considered a late-seral species more tolerant of shade, limber pine is a shade-intolerant pioneer species able to establish on infertile sites.Every autumn, seeds of the two species were collected from high- (3370 m–3570 m) and low-provenance (2910–3240 m) sources close to the experimental sites and sown in our plots. A subset of plots were heated and another subset watered over the summer months to offset the effects of warming. Across five years, we found that seeds originating from low elevation recruited more strongly for both species, although this provenance difference diminished by the fourth year for Engelmann spruce, likely due to small sample sizes. Despite the recruitment of low-provenance seed, warming treatments decreased recruitment at all elevations. Combining this with the likeliness and availability of lower-quality, high provenance seed moving upslope at the treeline, tree migration into the alpine may be slowed. Overall, our findings suggest that the hardier limber pine is likely to become a more significant species in subalpine forest communities in the future, while the more sensitive Engelmann spruce may experience range contraction.

54 ENVIRONMENTAL SCIENCES↗