Search NASA⌕ Search

SEARCH · Search NASA

Results for “Knowledge Modeling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Learning of networked spreading models from noisy and incomplete data

Recent years have seen a lot of progress in algorithms for learning parameters of spreading dynamics from both full and partial data. Some of the remaining challenges include model selection under the scenarios of unknown network structure, noisy data, missing observations in time, as well as an efficient incorporation of prior information to minimize the number of samples required for an accurate learning. Here, in this work, we introduce a universal learning method based on a scalable dynamic message-passing technique that addresses these challenges often encountered in real data. The algorithm leverages available prior knowledge on the model and on the data, and reconstructs both network structure and parameters of a spreading model. We show that a linear computational complexity of the method with the key model parameters makes the algorithm scalable to large network instances.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Bayesian stability and force modeling for uncertain machining processes

Accurately simulating machining operations requires knowledge of the cutting force model and system frequency response. However, this data is collected using specialized instruments in an ex-situ manner. Bayesian statistical methods instead learn the system parameters using cutting test data, but to date, these approaches have only considered milling stability. This paper presents a physics-based Bayesian framework which incorporates both spindle power and milling stability. Initial probabilistic descriptions of the system parameters are propagated through a set of physics functions to form probabilistic predictions about the milling process. The system parameters are then updated using automatically selected cutting tests to reduce parameter uncertainty and identify more productive cutting conditions, where spindle power measurements are used to learn the cutting force model. The framework is demonstrated through both numerical and experimental case studies. Results show that the approach accurately identifies both the system natural frequency and cutting force model.

42 ENGINEERING↗
DOE OSTI2025

BioPortal: an open community resource for sharing, searching, and utilizing biomedical ontologies

Abstract BioPortal (https://bioportal.bioontology.org) is the world’s most comprehensive repository of biomedical ontologies. It provides infrastructure for finding, sharing, searching, and utilizing biomedical ontologies. Launched in 2005, BioPortal now includes 1549 ontologies (1182 of them public). Its open, freely accessible website enables anyone (i) to browse the ontology library, (ii) to search for terms across ontologies, (iii) to browse mappings between terms, (iv) to see popularity ratings and recommendations on which ontologies are most relevant to their use cases, (v) to annotate text with ontology terms, (vi) to submit an ontology, and (vii) to request ontology changes. The library of ontologies can be accessed programmatically via a REST application programming interface (API). Recent enhancements include a BioPortal knowledge graph that integrates knowledge from multiple ontologies; a unified data model for interoperability with other knowledge sources; ontology popularity ratings and recommendations for relevant ontologies; and the ability to request ontology changes via a simple user interface that automatically converts user change requests to GitHub Pull Requests that specify the edits that will be made to the ontology upon approval.

Vendetti, Jennifer↗
DOE OSTI2025

Advancing Urban Water Resilience: Coproducing Knowledge through Civic–Academic Global Partnerships on Water and Climate

As extreme weather events become more pronounced, the vulnerabilities associated with the urban water supply and wastewater systems in megacities are intensified in multiple interconnected dimensions. These multifaceted water challenges can benefit from enhanced cross-sectoral collaboration and sharing of critical knowledge, which are essential for sustainable and adaptive water governance frameworks. In this context, the Megacity Alliance for Water and Climate (MAWAC)–Europe and North America Region (ENAR) Working Group convened a workshop in March 2023, followed by a subsequent workshop in London, United Kingdom, from 11 to 13 September 2024. These workshops aimed to investigate and devise solutions for the cascading hazards with water systems. The solutions examined various aspects focused on climate adaptation and mitigation, stormwater management, and the governance of water and wastewater systems. Additionally, discussions highlighted the importance of community engagement, economic considerations, equity, and effective communication in addressing these pressing challenges. Over the course of 3 days, experts from academia, government agencies, and industry engaged in meaningful discussions on digital modeling for integrated water management, climate-informed urban planning, and public–private–academic partnerships (Fig. 1). Case studies from cities such as New York, Los Angeles, London, Paris, and Chicago highlighted innovative governance strategies for managing water and wastewater systems, promoting water reuse, planning infrastructure, and fostering stakeholder-driven and stakeholder-informed adaptation. The workshop participants emphasized the need for data-driven decision-making, scalable governance models, and knowledge-sharing networks to enhance urban water governance for sustainability and resilience. This workshop report presents the key takeaways from the 3-day convening, providing a roadmap for integrating scientific research, policy frameworks, and emerging technologies to address water challenges faced by megacities.

Hydrologic models↗
DOE OSTI2026

Sim-to-real supervised domain adaptation for radioisotope identification

Machine learning has the potential to improve the speed and reliability of radioisotope identification using gamma spectroscopy. However, meticulously labeling an experimental dataset for training is often prohibitively expensive, while training models purely on synthetic data is risky due to the domain gap between simulated and experimental measurements. In this research, we demonstrate that supervised domain adaptation can substantially improve the performance of radioisotope identification models by transferring knowledge between synthetic and experimental data domains. We consider two domain adaptation scenarios: (1) a simulation-to-simulation adaptation, where we perform multi-label proportion estimation using simulated high-purity germanium detectors, and (2) a simulation-to-experimental adaptation, where we perform multi-class, single-label classification using measured spectra from handheld lanthanum bromide (LaBr) and sodium iodide (NaI) detectors. We begin by pretraining a spectral classifier on synthetic data using a custom transformer-based neural network. After subsequent fine-tuning on just 64 labeled experimental spectra, we achieve a test accuracy of 96% in the sim-to-real scenario with a LaBr detector, far surpassing a synthetic-only baseline model (75%) and a model trained from scratch (80%) on the same 64 spectra. Furthermore, we demonstrate that domain-adapted models learn more human-interpretable features than experiment-only baseline models. Overall, our results highlight the potential for supervised domain adaptation techniques to bridge the sim-to-real gap in radioisotope identification, enabling the development of accurate and explainable classifiers even in real-world scenarios where access to experimental data is limited.

Lalor, Peter W.↗
DOE OSTI2025

Modification and analysis of context-specific genome-scale metabolic models: methane-utilizing microbial chassis as a case study

ABSTRACT Context-specific genome-scale model (CS-GSM) reconstruction is becoming an efficient strategy for integrating and cross-comparing experimental multi-scale data to explore the relationship between cellular genotypes, facilitating fundamental or applied research discoveries. However, the application of CS modeling for non-conventional microbes is still challenging. Here, we present a graphical user interface that integrates COBRApy, EscherPy, and RIPTiDe, Python-based tools within the BioUML platform, and streamlines the reconstruction and interrogation of the CS genome-scale metabolic frameworks via Jupyter Notebook. The approach was tested using -omics data collected for Methylotuvimicrobium alcaliphilum 20Z R , a prominent microbial chassis for methane capturing and valorization. We optimized the previously reconstructed whole genome-scale metabolic network by adjusting the flux distribution using gene expression data. The outputs of the automatically reconstructed CS metabolic network were comparable to manually optimized i IA409 models for Ca-growth conditions. However, the CS model questions the reversibility of the phosphoketolase pathway and suggests higher flux via primary oxidation pathways. The model also highlighted unresolved carbon partitioning between assimilatory and catabolic pathways at the formaldehyde-formate node. Only a very few genes and only one enzyme with a predicted function in C1 metabolism, a homolog of the formaldehyde oxidation enzyme ( fae1-2 ), showed a significant change in expression in La-growth conditions. The CS-GSM predictions agreed with the experimental measurements under the assumption that the Fae1-2 is a part of the tetrahydrofolate-linked pathway. The cellular roles of the tungsten (W)-dependent formate dehydrogenase ( fdhAB ) and fae homologs ( fae1-2 and fae3 ) were investigated via mutagenesis. The phenotype of the f dhAB mutant followed the model prediction. Furthermore, a more significant reduction of the biomass yield was observed during growth in La-supplemented media, confirming a higher flux through formate. M. alcaliphilum 20Z R mutants lacking fae1-2 did not display any significant defects in methane or methanol-dependent growth. However, contrary to fae1, the fae1-2 homolog failed to restore the formaldehyde-activating enzyme function in complementation tests. Overall, the presented data suggest that the developed computational workflow supports the reconstruction and validation of CS-GSM networks of non-model microbes. IMPORTANCE The interrogation of various types of data is a routine strategy to explore the relationship between genotype and phenotype. An efficient approach for integrating and cross-comparing experimental multi-scale data in the context of whole-genome-based metabolic network reconstruction becomes a powerful tool that facilitates fundamental and applied research discoveries. The present study describes the reconstruction of a context-specific (CS) model for the methane-utilizing bacterium, Methylotuvimicrobium alcaliphilum 20Z R . M. alcaliphilum 20Z R is becoming an attractive microbial platform for the production of biofuels, chemicals, pharmaceuticals, and bio-sorbents for capturing atmospheric methane. We demonstrate that this pipeline can help reconstruct metabolic models that are similar to manually curated networks. Furthermore, the model is able to highlight previously overlooked pathways, thus advancing fundamental knowledge of non-model microbial systems or promoting their development toward biotechnological or environmental implementations.

Kulyashov, M. A.↗
DOE OSTI2026

Weak-charge form-factor determination at the electron-ion collider

Determining the weak charge form factor, 𝐹 𝑊 ⁡(𝑄 2 ), of nuclei over a continuous range of momentum transfers, 0 ≲ 𝑄 2 ≲ 0.1 GeV 2 , is essential for mapping out the distribution of neutrons in nuclei. The neutron density distribution has significant implications for a broad range of areas, including studies of nuclear structure, neutron stars, and physics beyond the Standard Model. Currently, our knowledge of 𝐹 𝑊 ⁡(𝑄 2 ) comes primarily from fixed target experiments that measure the parity-violating asymmetry in coherent elastic electron-ion scattering. Fixed target experiments, such as CREX and PREX-1,2, have provided high-precision weak charge form factor extractions for the 48 Ca and 208 Pb nuclei, respectively. However, a major limitation of fixed target experiments is that they each provide data only at a single value of 𝑄 2 . With the proposed electron-ion collider (EIC) on the horizon, we explore its potential to impact the determination of the weak charge form factor. While it cannot compete with the precision of fixed target experiments, it can provide data over a wide and continuous range of 𝑄 2 values, and for a wide variety of nuclei. We show that with data corresponding to an integrated luminosity of ℒ ∼ 500/𝐴 fb −1 , where 𝐴 is the nucleus atomic weight, the EIC can significantly impact constraints by lifting degeneracies in theoretical models of the neutron density distribution. Ensuring EIC detector coverage at low 𝑄 2 and large negative pseudorapidities will be essential for such 𝐹 𝑊 ⁡(𝑄 2 ) measurements.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

A machine learning pipeline for identifying infiltration managed aquifer recharge locations from satellite imagery in the San Joaquin Valley, California

This study focuses on an agricultural region in California’s Central Valley, USA, where Managed Aquifer Recharge (MAR) is widely implemented to mitigate groundwater depletion under increasing water demand and climate variability. A deep learning and machine learning framework was developed to identify infiltration-MAR locations using satellite imagery and environmental data. The framework integrates surface water detection from Sentinel-2 imagery, geospatial delineation of water bodies, spatiotemporal tracking of water body dynamics, and supervised classification using meteorological, environmental, and topographic variables. The framework was applied to a 2379 km² study area southwest of Fresno, where 765 water bodies were detected, including 139 identified MAR sites based on publicly available datasets and expert knowledge. The classification model achieved an accuracy of 0.94 and an F1 score of 0.85. Feature importance analysis indicates that cropland, normalized difference vegetation index (NDVI), and evaporation are among the most influential predictors for infiltration-MAR. Notably, the framework suggests that engineered water management in infiltration-MAR systems can disrupt or even reverse the expected positive correlation between surface water extent and precipitation. These findings provide physically interpretable insights into the characteristics of existing infiltration-MAR facilities and demonstrate the potential of the proposed framework as a reproducible, interpretable, and potentially transferable tool for data-driven infiltration-MAR identification and inventory development under growing climatic and hydrological uncertainty.

Classification↗
DOE OSTI2026

A critical review on additive manufacturing of refractory alloys from a data analytics perspective- beyond nickel-based superalloys

Refractory alloys (RAs) are promising materials due to their exceptional physicochemical properties, but most research remains at the laboratory scale. For broader adoption, advancements in manufacturing are essential. Because their high stability makes conventional methods like machining and casting difficult, additive manufacturing (AM) is emerging as an effective approach for fabricating refractory alloy components. However, AM's repeated non-equilibrium thermal cycles introduce undesired features (e.g. defects, anisotropic microstructures, and residual stresses), which are magnified due to RAs’ unique properties. This paper comprehensively reviews the state-of-the-art methods of AM for refractory alloys. It explores data analytics techniques to establish design rules based on multi-fidelity experimental and computational methods. Furthermore, it investigates integrated, collaborative efforts to harmonise standalone databases, information, knowledge, and predictive models at multi-physics, multi-stage, and multi-scale. Unlike the existing literature that focuses primarily on material systems or process fundamentals, this work provides an integrated perspective on AM of refractory alloys from a data analytics standpoint, highlighting the roles of integrated computational materials engineering (ICME), verification, validation, and uncertainty quantification (VV&UQ), and digital twin-driven qualification in overcoming data scarcity and accelerating rapid qualification.

Additive manufacturing↗
DOE OSTI2026

Knowledge Graph for End-to-End Traceability of an Integrated Human-Earth System Model

Integrated human-Earth system models inform energy-water-land system dynamics and policies, yet their results are difficult to trace through input-data, model structure, scenario configurations, and solved outputs. Because this information is siloed across disconnected artifacts, process-based IAMs have historically lacked a unified, queryable representation. Such lack of traceability prevents researchers from systematically isolating the multi-sector drivers of complex outcomes (such as tracing water-scarcity results back to distant energy-system dynamics) or conducting holistic uncertainty attribution across hundreds of interacting parameters. To address this concern, our work documents the software engineering process of a knowledge graph that unifies these four layers for the Global Change Analysis Model (GCAM-USA_Reference scenario, GCAM v9.1). The graph was built as a relational property graph in DuckDB from the run’s own artifacts: the input-preparation dependency map (gcamdata chunk map), the model’s XML input files, the run configuration, and the results database (BaseX), successfully mapping the model’s declared structure. The resulting graph comprises 204,321 nodes and 1,687,814 edges across 16 node types and 15 edge types, with approximately 16.3 million time-series values stored separately to maintain structural efficiency. To ensure representation fidelity, every edge carries an epistemic-status annotation recording the warrant for the relationship (structural, provenance, dependency, or model-derived), and a machine-readable provenance ledger classifying the origin of every schema element. Evaluation against a fixed five-benchmark suite with locked baselines reports zero structural orphans, zero dangling edge endpoints, and 100% of output-producing technologies traceable to raw input files. Two interactive interfaces present the graph, including a serverless browser application built on DuckDB-Wasm. By establishing the first end-to-end provenance framework for an IAM, this work enables researchers and scientists to systematically audit complex policy scenarios, debug model structures, and trace policy-relevant outputs to their data origins in real time.

Artifical Intelligence↗
DOE OSTI2025

Fully implicit crystal plasticity models representing orientations with modified Rodrigues parameters

Here, this work describes a crystal plasticity formulation combining several mathematical, numerical, and implementation choices to produce a highly efficient model. Specifically, the key choices in the implementation are (1) representing orientations with modified Rodrigues parameters, (2) implementing a fully coupled implicit time integration for the elastic stretch, the crystal orientations, and the model internal variables, (3) implementing the model in the NEML2 constitutive modeling framework, based on PyTorch, to vectorize the calculations and port the computation to GPUs and other hardware accelerators, and (4) an exact implementation of the consistent tangent matrix, even for arbitrary coupling to other field variables beyond the displacements, like temperature, neutron fluence, etc. The first two features of the model are, to our knowledge, novel. The paper considers each of these choices individually as well as the final model as a whole. This includes a full description of modified Rodrigues parameters, their advantages over other representations of orientations, the mathematical formulae and tools required to implement a model with modified Rodrigues parameters, and a detailed description of the geometry of the space of modified Rodrigues parameters (in an appendix). It also includes a description of a fully implicit time integration scheme for the orientations and the advantages in representing orientations with modified Rodrigues parameters in implementing such a model. The work then assess, via numerical examples, the advantages of fully coupled implicit time integration versus more common decoupled and explicit time integration schemes. These studies demonstrate the computational advantages of fully coupled integration versus other time integration algorithms, though the performance of the competing models depends on the complexity of the underlying single crystal model. The study concludes by demonstrating that the choice of time integration method affects the sharpness of the predicted texture, with explicit methods for integrating the orientations overestimating texture sharpness and implicit methods underestimating texture sharpness.

Crystal plasticity↗
DOE OSTI2024

Impacts of photovoltaic solar energy on soil carbon: A global systematic review and framework

Globally, solar energy is anticipated to be the primary source of electricity as early as 2050, and the greatest additions in capacity are currently in the form of large, ground-mounted photovoltaic solar energy facilities (GPVs). Growing interest lies in understanding and anticipating opportunities to increase soil carbon sequestration across the footprint and perimeter of both conventional and multi-use GPVs (e.g., ecovoltaics, agrivoltaics, and rangevolatics), especially as operators increasingly deputize as land managers. To date, studies on the relationship between soils and PV solar energy are limited to unique, localized sites. This study employed a systematic review to (i) identify a global corpus of 18 studies on interactions between GPVs and soils, (ii) collect and characterize 113 soil and soil-related experimental variables interacting with GPVs from this corpus, and (iii) synthesize trends among these experimental variables. Next, this study combined data from the systematic review with an iterative, knowledge co-production approach to produce a conceptual model for the study of soil and GPV interactions that applies to multiple installation types, scales, and contexts where GPVs are deployed, and identified research opportunities, threats, and priorities. This study's baseline understanding, conceptual model, and co-produced knowledge confer unique insight into the feasibility of combining soil carbon sequestration with the climate change mitigation potential of PV solar energy.

agrivoltaics↗
DOE OSTI2024

Multi-Fidelity Modeling and Control for Building Temperature Control

The ability to control energy loads such as a building's heating, ventilation, and air conditioning (HVAC) system can help facilitate increased penetration of variable renewable energy sources into the electric grid. To be able to control these HVAC systems more effectively, detailed simulations of the corresponding building physics is becoming increasingly important. These detailed simulations can be complex, nonlinear, and can require immense computational power when used in an advanced control method such as model predictive control (MPC), prompting the need to explore less computationally intensive strategies. In this work, a multi-fidelity approach is proposed to combine samples from a complex, high-fidelity model with a simple, low-fidelity model within the MPC control loop. More specifically, the parameters of a reduced-order, linear building model are periodically updated with knowledge from its high-fidelity counterpart - an EnergyPlus model - in an online fashion using a Gaussian Process surrogate model. Hence, highly accurate predictions of current and future conditions in a building are maintained with a substantially reduced computational burden compared to using the high-fidelity models alone. In other words, this linear parameter varying model preserves the low computational requirements of a low-order linear model while accurately modeling a building's dynamics. This allows a building controller to take highly informed actions without requiring a large computational budget.

building modeling↗
DOE OSTI2025

A Model Based Approach to Extract Health Information from Textual Data

In current nuclear power plants (NPPs) a large amount of condition-based data is being generated and stored to assess and monitor component health and performance. The format of this data can be either numeric (e.g., pump vibration data) or textual (e.g., condition report which assess component health). While assessing component health from numeric data can be performed with a large variety of methods, the extraction of information from textual data still remains a challenge. Natural language processing (NLP) methods are starting to be deployed in current NPPs mainly to filter out incident reports (IRs) that are not safety related by employing supervised machine learning methods. However, these methods do not really provide the quantitative information that might be contained in IRs. This paper presents an approach to extract information from textual data (e.g., from IRs, maintenance reports) that is based on NLP data analytics methods coupled with model-based system engineer (MBSE) models. NLP methods are employed to perform syntactic and semantic analyses. Syntactic analysis analyzes the grammatical structure of a sentence; such analysis includes: part of speech (POS) tagging (i.e., identification of grammatic elements of each string - e.g., nouns, verbs), named entity recognition (i.e., identification of text entities - e.g., names, dates, events), and relation extraction (e.g., coreference resolution). On the other hand, semantic analysis is designed to analyze the logic structure of a sentence. Through a specific set of rules, our methods can identify whether a sentence contains health information of a component (e.g., degraded performance, anomaly behavior) or the causal relationship between two events (i.e., a cause-effect pair). An innovative element of our approach is that semantic analysis relies on MBSE models to identify links between textual elements. MBSE are diagrams designed to represent system and component dependencies (from both a form and functional point of view). In our approach, MBSE models emulate system engineer knowledge about component/system architecture. This paper presents in detail how the integration of NLP methods and MBSE models is performed. Few analysis examples focusing on centrifugal pumps are presented.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Improvement and generalization of ABCD method with Bayesian inference

To find New Physics or to refine our knowledge of the Standard Model at the LHC is an enterprise that involves many factors, such as the capabilities and the performance of the accelerator and detectors, the use and exploitation of the available information, the design of search strategies and observables, as well as the proposal of new models. We focus on the use of the information and pour our effort in re-thinking the usual data-driven ABCD method to improve it and to generalize it using Bayesian Machine Learning techniques and tools. We propose that a dataset consisting of a signal and many backgrounds is well described through a mixture model. Signal, backgrounds and their relative fractions in the sample can be well extracted by exploiting the prior knowledge and the dependence between the different observables at the event-by-event level with Bayesian tools. We show how, in contrast to the ABCD method, one can take advantage of understanding some properties of the different backgrounds and of having more than two independent observables to measure in each event. In addition, instead of regions defined through hard cuts, the Bayesian framework uses the information of continuous distribution to obtain soft-assignments of the events which are statistically more robust. To compare both methods we use a toy problem inspired by pp\to hh\to b\bar b b \bar b p p → h h → b b ‾ b b ‾ , selecting a reduced and simplified number of processes and analysing the flavor of the four jets and the invariant mass of the jet-pairs, modeled with simplified distributions. Taking advantage of all this information, and starting from a combination of biased and agnostic priors, leads us to a very good posterior once we use the Bayesian framework to exploit the data and the mutual information of the observables at the event-by-event level. We show how, in this simplified model, the Bayesian framework outperforms the ABCD method sensitivity in obtaining the signal fraction in scenarios with 1% and 0.5% true signal fractions in the dataset. We also show that the method is robust against the absence of signal. We discuss potential prospects for taking this Bayesian data-driven paradigm into more realistic scenarios.

Alvarez, Ezequiel↗
DOE OSTI2025

RingX: Scalable Parallel Attention for Long-Context Learning on HPC

The attention mechanism has become foundational for remarkable AI breakthroughs since the introduction of the Transformer, driving the demand for increasingly longer context to power frontier models such as large-scale reasoning language models and high-resolution image/video generators. However, its quadratic computational and memory complexities present substantial challenges. Current state-of-the-art parallel attention methods, such as ring attention, are widely adopted for long-context training but utilize a point-to-point communication strategy that fails to fully exploit the capabilities of modern HPC network architectures. In this work, we propose ringX, a scalable family of parallel attention methods optimized explicitly for HPC systems. By enhancing workload partitioning, refining communication patterns, and improving load balancing, ringX achieves up to 3.4 × speedup compared to conventional ring attention on the Frontier supercomputer. Optimized for both bi-directional and causal attention mechanisms, ringX demonstrates its effectiveness through training benchmarks of a Vision Transformer (ViT) on a climate dataset and a Generative Pre-Trained Transformer (GPT) model, Llama3 8B. Our method attains an end-to-end training speedup of approximately 1.5 × in both scenarios. To our knowledge, the achieved 38% model FLOPs utilization (MFU) for training Llama3 8B with a 1M-token sequence length on 4,096 GPUs represents one of the highest training efficiencies reported for long-context learning on HPC systems. Our code implementation is available at https://github.com/jqyin/ringX-attention.

Yin, Junqi [ORNL] (ORCID:0000000338435520)↗
DOE OSTI2026

Dispersion Loss Counteracts Embedding Condensation and Improves Generalization in Small Language Models

Large language models (LLMs) achieve remarkable performance through ever-increasing parameter counts, but scaling incurs steep computational costs. To better understand LLM scaling, we study representational differences between LLMs and their smaller counterparts, with the goal of replicating the representational qualities of larger models in smaller models. We observe a geometric phenomenon which we term embedding condensation, where token embeddings collapse into a narrow cone-like subspace in some language models. Through systematic analyses across multiple Transformer families, we show that small models such as GPT2 and Qwen3-0.6B exhibit severe condensation, whereas larger models such as GPT2-x1 and Qwen3-32B are more resistant to this phenomenon. Additional observations show that embedding condensation is not reliably mitigated by knowledge distillation from larger models. To fight against it, we formulate a dispersion loss that explicitly encourages embedding dispersion during training. Experiments demonstrate that it mitigates condensation, recovers dispersion patterns seen in larger models, and yields performance gains across 10 benchmarks. We believe this work offers a principled path toward improving smaller Transformers without additional parameters.

Xiao, Xi [ORNL] (ORCID:0009000009316982)↗
DOE OSTI2026

Lessons Learned from Ecosystem-Scale Experimental Field Studies (Workshop Report)

Efforts to understand and predict ecosystem responses to environmental change require long-term, large-scale, spatially representative experiments and observations that capture natural variability, test predictive models, and generate transferable knowledge. Such studies are indispensable for unraveling the complexities of terrestrial ecosystems and their responses to disturbances and evolving environmental conditions, while generating the data necessary for developing mechanistic models and predictive tools that inform decision-making processes. Having a rich history of designing and executing large-scale ecosystem experiments, the U.S. Department of Energy’s Environmental System Science program convened a workshop in January 2025 that brought together leaders in the field to distill critical lessons from decades of experience in large-scale experiments. The workshop aimed to (1) provide an ecosystem experiment primer for best practices, thus ensuring a high scientific return on investment for funding agencies, and (2) offer a robust framework for the design and management of future research initiatives. This report synthesizes insights and experiences from workshop participants and is structured to capture the entire research life cycle, from goal setting and design to operations, adaptive management, team dynamics, collaborations, and the often overlooked aspect of decommissioning. By synthesizing decision-making and lessons learned across diverse research approaches, the report aims to provide a template of essential factors to consider when designing successful long-term, large-scale ecosystem experiments.

54 ENVIRONMENTAL SCIENCES↗