Search NASA⌕ Search

SEARCH · Search NASA

Results for “RoBERTa”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

An Introduction to Word Embeddings and Language Models

Language models have advanced at a phenomenal pace over the past decade. This document provides a short introduction to terminology, word embeddings (aka low-dimensional representations), and popular large-scale language models (LMs). Word embeddings are used to represent words as numerical vectors and are context-independent, meaning a word can only have a single representation (e.g., club can only be club sandwich, not golf club ). Language models can determine the probability of a given sequence of words occurring in a sentence and can provide context to distinguish between words and phrases that sound similar. LMs are context-dependent (e.g., club can be club sandwich or golf club ) and largely fall in two main classes – autoregressive and autoencoding models. Autoregressive models are pretrained on the classic language modeling task: guess the next token having read all the previous ones. Those models can be fine-tuned and achieve great results on many tasks, the most natural application is text generation. A typical example of such models is GPT, but others include GPT-2, GPT-3, CTLR, TRANSFORMER-XL, REFORMER, XLNET. Autoencoding models are pretrained by corrupting the input tokens in some way and trying to reconstruct the original sentence. They can be fine-tuned and achieve great results on many tasks such as text generation, but their most natural application is sentence classification or token classification. A typical example of such models is BERT, but others include ROBERTA, ALBERT, XML, XML-ROBERTA, FLAUBERT AND LONGFORMER.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

VWC-BERT: Scaling Vulnerability–Weakness–Exploit Mapping on Modern AI Accelerators

Defending cybersystems needs accurate mapping of software and hardware vulnerabilities to generalized descriptions of weaknesses, and weaknesses to exploits. These mappings enable cyber defenders to build plans for effective defense and assessment of potential risks to a cybersystem. With close to 170k vulnerabilities, manual mapping is not a feasible option. However, automated mapping is challenging due to limited training data, computational intractability, and limitations in computational natural language processing. Tools based on breakthroughs in Transformer-based language models have been demonstrated to classify vulnerabilities with high accuracy. We make three key contributions in this paper: (1) We present a new framework, \VWCBERT, that augments the Transformer-based hierarchical multi-class classification framework of Das et al. (\textsc{V2W-BERT}) with the ability to map weaknesses to exploits. (2) We implement \VWCBERT~ on modern AI accelerator platforms using two data parallel techniques for the pre-training phase and demonstrate nearly linear speedups across NVIDIA and Graphcore accelerator platforms. We observe nearly linear speedups for up to 16 V100 and 8 A100 GPUs, and about 3.4$\times$ speedup for A100 relative to V100 GPUs. We also observe excellent speedups on Graphcore, with $5.7\times$ speedup on 128 IPUs relative to 16 IPUs. Enabled by scaling, we also demonstrate higher accuracy using a larger language model, RoBERTa-Large. We show up to 87\% accuracy for strict and up to 98\% accuracy for relaxed classification. (3) We develop a novel parallel link manager for the link prediction phase and demonstrate up to 21$\times$ speedup with 16 V100 GPUs relative to one V100 GPU, and thus reducing the runtime from 2.5 hours to 10 minutes. We believe that generalizability and scalability of \VWCBERT~ will benefit both the theoretical development and practical deployment of novel cyberdefense solutions and vulnerability classification.

Das, Siddhartha Shankar↗
DOE OSTI2024

Chiral Teropyrenes: Synthesis, Structure, and Spectroscopic Studies

Abstract We present the inaugural synthesis of a chiral teropyrene achieved through a four‐fold alkyne benzannulation catalyzed by InCl 3 , resulting in good yields. The product underwent thorough characterization using FT‐Raman and FT‐IR spectroscopies, demonstrating a close agreement with calculated spectra. X‐ray crystallographic analysis unveiled a notable twist in the molecule‘s backbone, with an end‐to‐end twist angle of 51°, consistent with computational predictions. Experimentally determined enantiomeric inversion barriers revealed a significant energy barrier of 23 kcal/mol, facilitating the isolation of enantiomers for analysis by circular dichroism (CD) and circularly polarized luminescence (CPL) spectroscopies. These findings mark significant strides in the synthesis and characterization of chiral teropyrenes, offering insights into their structural and spectroscopic properties.

Chemistry↗
DOE OSTI2024

Defining Golden Batches in Biomanufacturing Processes From Internal Metabolic Activity to Detect Process Changes That May Affect Product Quality

ABSTRACT Cellular metabolism plays a role in the observed variability of a drug substance's Critical Quality Attributes (CQAs) made by biomanufacturing processes. Therefore, here we describe a new approach for monitoring biomanufacturing processes that measures a set of metabolic reaction rates (named Critical Metabolic Parameters (CMP) in addition to the macroscopic process conditions currently being used as Critical Process Parameters (CPP) for biomanufacturing. Constraint‐based systems biology models like Flux Balance Analysis (FBA) are used to estimate metabolic reaction rates, and metabolic rates are used as inputs for multivariate Batch Evolution Models (BEM). Metabolic activity was reproducible among batches and could be monitored to detect a deliberately induced macroscopic process shift (i.e., temperature change). The CMP approach has the potential to enable “golden batches” in biomanufacturing processes to be defined from the internal metabolic activity and to aid in detecting process changes that may impact the quality of the product. Overall, the data suggested that monitoring of metabolic activity has promise for biomanufacturing process control.

Biotechnology & Applied Microbiology↗
DOE OSTI2022

Simple Dipyrrin Analogues of Prodigiosin for Use as Colistin Adjuvants

Multidrug resistant (MDR) bacteria are an increasing public health problem. One promising alternative to the development of new antibiotics is the use of antibiotic adjuvants, which would allow the continued use of FDA-approved antibiotics that have been rendered ineffective due to resistance. In this study, we report a series of dipyrrins and pyrrole derivatives designed as analogues of prodigiosin and obatoclax, several of which potentiate the activity of colistin against Klebsiella pneumoniae, with lead compounds also potentiating colistin against Acinetobacter baumannii and Pseudomonas aeruginosa.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

The Zooplankton International Geospatial dataset: A global repository of spatiotemporal freshwater zooplankton community composition data from lakes and reservoirs to support ecological research

Zooplankton transfer substantial energy in aquatic food webs and are used as indicators of environmental change. Syntheses of zooplankton community dynamics globally require datasets that span a wide range of environmental gradients; however, these datasets are limited due to methodological differences across programs, taxonomic inconsistencies, and a lack of standardized metadata. To reconcile these challenges, we created the Zooplankton International Geospatial (ZIG) dataset, which includes original zooplankton, water physical and chemical variables, and lake morphometric data from 311 inland lakes and reservoirs. ZIG includes waterbodies ranging in size from 0.005 to 82,100 km2 and spanning broad latitudinal (−47.26 to 64.90) and longitudinal ranges (−165.04 to 176.53). Temporal coverage for individual waterbodies ranges between 1 and 60 yr with sampling frequency ranging from annually to weekly. With its extensive coverage and content, we consider ZIG to be a cornerstone for future investigations of global scale lake biodiversity change.

Figary, Stephanie [Cornell University, Ithaca, NY]↗
DOE OSTI2025

Non-invertible defects from the Conway SCFT to K3 sigma models. Part I. General results

We initiate the study of supersymmetry-preserving topological defect lines (TDLs) in the Conway moonshine module V f♮ . We show that the tensor category of such defects, under suitable assumptions, admits a surjective but non-injective ring homomorphism into the ring of $\mathbb{Z}$-linear maps of the Leech lattice into itself. This puts strong constraints on possible defects and their quantum dimensions. We describe a simple construction of non-invertible TDLs from orbifolds of holomorphic (super)vertex operator algebras, which yields non-trivial examples of TDLs satisfying our main theorem. We conjecture a correspondence between four-plane-preserving TDLs in V f♮ and supersymmetry-preserving TDLs in K3 non-linear sigma models, which extends the correspondence between symmetry groups to the level of tensor category symmetry. We establish evidence for this conjecture by constructing non-invertible TDLs in special K3 non-linear sigma models.

Conformal Field Models in String Theory↗
DOE OSTI2023

Structural comparison of allophycocyanin variants reveals the molecular basis for their spectral differences

Abstract Allophycocyanins are phycobiliproteins that absorb red light and transfer the energy to the reaction centers of oxygenic photosynthesis in cyanobacteria and red algae. Recently, it was shown that some allophycocyanins absorb far-red light and that one subset of these allophycocyanins, comprising subunits from the ApcD4 and ApcB3 subfamilies (FRL-AP), form helical nanotubes. The lowest energy absorbance maximum of the oligomeric ApcD4-ApcB3 complexes occurs at 709 nm, which is unlike allophycocyanin (AP; ApcA-ApcB) and allophycocyanin B (AP-B; ApcD-ApcB) trimers that absorb maximally at ~ 650 nm and ~ 670 nm, respectively. The molecular bases of the different spectra of AP variants are presently unclear. To address this, we structurally compared FRL-AP with AP and AP-B, performed spectroscopic analyses on FRL-AP, and leveraged computational approaches. We show that among AP variants, the α-subunit constrains pyrrole ring A of its phycocyanobilin chromophore to different extents, and the coplanarity of ring A with rings B and C sets a baseline for the absorbance maximum of the chromophore. Upon oligomerization, the α-chromophores of all AP variants exhibit a red shift of the absorbance maximum of ~ 25 to 30 nm and band narrowing. We exclude excitonic coupling in FRL-AP as the basis for this red shift and extend the results to discuss AP and AP-B. Instead, we attribute these spectral changes to a conformational alteration of pyrrole ring D, which becomes more coplanar with rings B and C upon oligomerization. This study expands the molecular understanding of light-harvesting attributes of phycobiliproteins and will aid in designing phycobiliproteins for biotechnological applications.

Gisriel, Christopher J.↗
DOE OSTI2020

The SuperCam Instrument Suite on the NASA Mars 2020 Rover: Body Unit and Combined System Tests

The SuperCam instrument suite provides the Mars 2020 rover, Perseverance, with a number of versatile remote-sensing techniques that can be used at long distance as well as within the robotic-arm workspace. These include laser-induced breakdown spectroscopy (LIBS), remote time-resolved Raman and luminescence spectroscopies, and visible and infrared (VISIR; separately referred to as VIS and IR) reflectance spectroscopy. A remote micro-imager (RMI) provides high-resolution color context imaging, and a microphone can be used as a stand-alone tool for environmental studies or to determine physical properties of rocks and soils from shock waves of laser-produced plasmas. SuperCam is built in three parts: The mast unit (MU), consisting of the laser, telescope, RMI, IR spectrometer, and associated electronics, is described in a companion paper. The on-board calibration targets are described in another companion paper. Here we describe SuperCam’s body unit (BU) and testing of the integrated instrument. The BU, mounted inside the rover body, receives light from the MU via a 5.8 m optical fiber. The light is split into three wavelength bands by a demultiplexer, and is routed via fiber bundles to three optical spectrometers, two of which (UV and violet; 245–340 and 385–465 nm) are crossed Czerny-Turner reflection spectrometers, nearly identical to their counterparts on ChemCam. The third is a high-efficiency transmission spectrometer containing an optical intensifier capable of gating exposures to 100 ns or longer, with variable delay times relative to the laser pulse. This spectrometer covers 535–853 nm ( $105\text{--}7070~\text{cm}^{-1}$ Raman shift relative to the 532 nm green laser beam) with $12~\text{cm}^{-1}$ full-width at half-maximum peak resolution in the Raman fingerprint region. The BU electronics boards interface with the rover and control the instrument, returning data to the rover. Thermal systems maintain a warm temperature during cruise to Mars to avoid contamination on the optics, and cool the detectors during operations on Mars.Results obtained with the integrated instrument demonstrate its capabilities for LIBS, for which a library of 332 standards was developed. Examples of Raman and VISIR spectroscopy are shown, demonstrating clear mineral identification with both techniques. Luminescence spectra demonstrate the utility of having both spectral and temporal dimensions. Finally, RMI and microphone tests on the rover demonstrate the capabilities of these subsystems as well.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Geophysical methods reveal the soil architecture and subsurface stratigraphic heterogeneities across land-lake interfaces along Lake Erie

Abstract Purpose The land-lake interface is a unique zone where terrestrial and aquatic ecosystems meet, forming part of the Earth’s most geochemically and biologically active zones. The unique characteristics of this interface are yet to be properly understood due to the inherently high spatiotemporal variability of subsurface properties, which are difficult to capture with the traditional soil sampling methods. Geophysical methods offer non-invasive techniques to capture variabilities in soil properties at a high resolution across various spatiotemporal scales. Methods We combined electromagnetic induction (EMI), electrical resistivity tomography (ERT), and ground penetrating radar (GPR) with data from soil cores and in situ sensors to investigate hydrostratigraphic heterogeneities across land-lake interfaces along the western basin of Lake Erie. Results EMI revealed high spatial heterogeneities in ECa distribution across the land-lake interfaces, with higher values in the wetland and transition zones compared to the upland zone. Soil ECa maps matched soil maps from a public database with the hydric soil units delineated as high conductivity zones (ECa > 40 mS/m). ERT and GPR showed vertical variation in soil properties with clear stratigraphic boundaries, and correlation of ERT profiles with lithologs from piezometers revealed the stratigraphic units of silt–clay and till sequence down to 3.5 m depth which are consistent with the surficial geology of the study area. Conclusions These results validate the use of multiple geophysical methods for extrapolating soil properties and mapping stratigraphic structures at land-lake interfaces, thereby providing the missing information required to improve the earth system model (ESM) of coastal interfaces.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Litter removal impacts on soil biodiversity and eucalypt plantation development in the seasonal tropics

The little layer of tree plantations provides primary nutrients for uptake, buffers changes in soil moisture, and provides habitat and substrate to soil epigeic fauna. However, this layer in eucalypt plantations is often removed to reduce fuel load during the fire season in the Brazilian savanna (Cerrado). Therefore, it is necessary to quantify the effects of changes in litter dynamics on the function of these plantations, on key nutrient cycling processes and on epigeic fauna diversity and abundance. In two adjacent stands (one juvenile and one mature), the consequences of two years of litter removal were quantified as monthly litterfall, leaf and fine wood litter decomposition, epigeic fauna abundance and diversity, soil biogeochemical variables, and tree diameter and basal area increments. Monthly litterfall rates in juvenile and mature stands did not change with litter removal over the study period. Annual litterfall ranged from 4.1 to 4.9 Mg ha -1 a -1 in litter removal plots and from 3.9 to 4.8 Mg ha -1 a -1 in control plots. Fine wood litter decomposition was slower in litter removal plots compared to controls, while leaf decomposition rates were similar in both. Two years of litter removal in the juvenile stand did not affect topsoil biogeochemical parameters but decreased available phosphorus at 20–40 cm depth relative to controls. In the mature stand, total cation exchange capacity (0–20 cm) was higher in controls (6.4 cmol c dm -3 ) relative to litter removal plots (6.3 cmol c dm -3 ), while soil moisture (0–40 cm depth) was lower in litter removal (25.45 m 3 m -3 ) compared to control plots (26 m 3 m -3 ) in the dry season. A non-metric multidimensional scaling ordination revealed an increased homogeneity in epigeic fauna where litter was removed. Litterfall, decomposition, diameter increment, four soil physical parameters and fourteen chemical parameters at 0–20 and 20–40 cm depth explained the differences in soil epigeic fauna composition between litter removal and control plots. Diameter increment decreased with litter removal only in the juvenile stand, which had reached its growth peak. The results indicate that removing excess litter to decrease fuel volume can alter soil biodiversity and edaphic conditions that negatively affect nutrient cycling and tree growth.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Photon Reconstruction in the Belle II Calorimeter Using Graph Neural Networks

We present the study of a fuzzy clustering algorithm for the Belle II electromagnetic calorimeter using Graph Neural Networks. We use a realistic detector simulation including simulated beam backgrounds and focus on the reconstruction of both isolated and overlapping photons. We find significant improvements of the energy resolution compared to the currently used reconstruction algorithm for both isolated and overlapping photons of more than 30% for photons with energies E γ < 0.5 GeV and high levels of beam backgrounds. Overall, the GNN reconstruction improves the resolution and reduces the tails of the reconstructed energy distribution and therefore is a promising option for the upcoming high luminosity running of Belle II.

Calorimeter↗
DOE OSTI2021

Development of a cell surface display system in Chlamydomonas reinhardtii

Cell-surface display systems are biotechnological techniques used to express heterologous proteins on the cell surface. Their application depends directly on the cell system used, as well as on the anchoring point for the surface displayed protein. To meet most application demands an inexpensive, safe, and scalable production platform, that reduces the economic barriers for large scale use is needed. Toward this goal, we screened three possible cell surface anchoring points in the green algae Chlamydomonas by fusing mVenus to prospective anchors moieties. The vectors harboring mVenus:anchor were screened for mVenus fluorescence and tested for cellular localization by confocal laser scanning microscopy. This strategy allowed the identification of two functional anchors, one for the cytoplasmic membrane using the MAW8 GPI-anchor signal, and one for the cell wall using the GP1 protein. We also exploited GP1 chemical and biological traits to release the fused proteins efficiently during cell wall shedding. Our work provides a foundation for surface engineering of C reinhardtii supporting both cell biology studies and biotechnology applications.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Linking soil biodiversity and ecosystem function in a Neotropical savanna

Conserving the remaining savanna ecosystems in the Brazilian savanna (Cerrado) — a global biodiversity hotspot that stores carbon and provides water to a large portion of South America — requires understanding the ecological processes maintaining their function. Nutrient cycling supports savanna function via plant litter production and decomposition by soil fauna, releasing nutrients for plant and soil organism uptake. Soil biodiversity and biogeochemistry linkages with litter dynamics in Neotropical savannas under a changing climate are poorly understood. Here, in this study, we combined two years of rainfall seasonality, leaf and wood litter production and decomposition with soil epigeic fauna abundance — the number of ground-surface dwelling invertebrates collected through pitfall traps — taxa richness, Shannon's diversity and Pielou's evenness, and 16 soil biogeochemical variables measured in 12 plots of preserved savanna. Rainfall seasonality modulated the mean soil epigeic fauna diversity and evenness across all plots, which were highest in the rainy season, in contrast to litterfall rates, which peaked in the dry season. In the dry season (April to September), the Formicidae family was the most abundant with 50% of all individuals, while in the rainy season (October to March), the Isoptera order was the most abundant with approximately 39% of individuals. Wood litter decomposition grouped with annual Hemiptera abundance, co-varying with soil epigeic fauna diversity and evenness per plot and against soil fertility variables. Leaf litter decomposition co-varied with the total epigeic fauna abundance and soil pH. We speculate that the specific need to decompose wood litter may be associated with a greater need for diversity than an abundance of soil epigeic fauna. Our work highlights the role of rainfall seasonality on soil biodiversity and physicochemistry, which is also tightly linked with litter production and decomposition. This study advances our understanding of the mechanisms governing nutrient cycling in savanna ecosystems on nutrient-impoverished soils, with implications for achieving sustainable conservation and restoration goals.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Carbon source-induced changes in the transcriptional landscape of Novacetimonas hansenii

Novacetimonas hansenii is a model organism for bacterial cellulose (BC) production, a biopolymer with broad industrial applications. To better understand its carbon source-dependent metabolism, we conducted the first time-resolved transcriptomic analysis of N. hansenii ATCC 23769 grown on five carbon sources (glucose, fructose, mannitol, galactose, and sucrose) across five time points (24–120h). Glucose, fructose, and mannitol supported growth and efficient BC production, whereas galactose and sucrose resulted in poor yields, likely due to limited transport and enzymatic capacity. Solid-state NMR revealed that cellulose crystallinity and polymorphism were carbon source-dependent, with reduced bcsD expression correlating with lower crystallinity. Transcriptomic profiling showed distinct metabolic responses. Cultures containing glucose and mannitol activated core metabolic pathways, fructose induced a delayed adaptive shift, and galactose and sucrose triggered stress-related gene expression. These results highlight the metabolic plasticity of N. hansenii and offer insights for enhancing BC production through carbon source optomization.

Bacterial cellulose↗
DOE OSTI2023

Redox and mineral controls on Fe and Ti isotopic fractionations during calc-alkaline magmatic differentiation

Titanium and Fe isotopic compositions of lavas from a calc-alkaline differentiation suite and corresponding mineral separates from the Rindjani Volcano, Indonesia show that Fe and Ti isotopic fractionations between minerals and melts are lower than those recorded in other suites at all stages of differentiation. Here, the limited isotopic fractionation for Ti is likely due to low-Ti magnetite and clinopyroxene being the dominant carriers of Ti in Rindjani lavas, as these minerals are thought to have limited equilibrium Ti isotopic fractionation relative to silicate magmas. Other magmatic differentiation suites controlled by removal of Ti-rich magnetite and characterized by a lesser role of clinopyroxene have larger Ti isotopic fractionations. This effect is an indirect consequence of the elevated Fe 3+ /Fe 2+ ratio of calc-alkaline magmas such as Rindjani, which promotes Fe 3+ incorporation into magnetite at the expense of Fe 2+ -Ti 4+ pairs, such that increased oxygen fugacity will subdue Ti isotopic fractionation in global magmatic series. Similarly, we find negligible Fe isotopic fractionation in Rindjani bulk rocks and mineral separates, unlike previous studies. This is also likely due to the oxidized nature of the Rindjani differentiation suite, which leads to similar Fe 3+ /Fe 2+ ratios in melt and minerals and decreases overall mineral-melt Fe fractionation factors. Paired Ti and Fe isotopic analyses may therefore represent a powerful tool to assess oxygen fugacity during differentiation, independent from Fe 3+ determinations of erupted samples.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗