Search NASA⌕ Search

SEARCH · Search NASA

Results for “dataset shift”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Latitudinal shifts of soil microbial biomass seasonality

Soil microbes ultimately drive the mineralization of soil organic carbon and thus ecosystem functions. We compiled a dataset of the seasonality of microbial biomass carbon (MBC) and developed a semi-mechanistic model to map monthly MBC across the globe. MBC exhibits an equatorially symmetric seasonality between the Northern and Southern Hemispheres. In the Northern Hemisphere, MBC peaks in autumn and is minimal in spring at low latitudes (<25°N), peaks in the spring and is minimal in autumn at mid-latitudes (25°N to 50°N), while peaks in autumn and is minimal in spring at high latitudes (>50°N). This latitudinal shift of MBC seasonality is attributed to an interaction of soil temperature, soil moisture, and substrate availability. The MBC seasonality is inconsistent with patterns of heterotrophic respiration, indicating that MBC as a proxy for microbial activity is inappropriate at this resolution. This study highlights the need to explicitly represent microbial physiology in microbial models. The interactive controls of environments and substrate on microbial seasonality provide insights for better representing microbial mechanisms in simulating ecosystem functions at the seasonal scale.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Cross-functional transferability in foundation machine learning interatomic potentials

The rapid development of foundation potentials (FPs) in machine learning interatomic potentials demonstrates the possibility for generalizable learning of the universal potential energy surface. The accuracy of FPs can be further improved by bridging the model from lower-fidelity datasets to high-fidelity ones. In this work, we analyze the challenge of this transfer learning (TL) problem within the CHGNet framework. We show that significant energy scale shifts and poor correlations between GGA and r 2 SCAN hinder cross-functional transferability. By benchmarking different TL approaches on the MP-r 2 SCAN dataset, we demonstrate the importance of elemental energy referencing in the TL of FPs. By comparing the scaling law with and without the pre-training on a low-fidelity dataset, we show that significant data efficiency can still be achieved through TL, even with a target dataset of sub-million structures. We highlight the importance of proper TL and multi-fidelity learning in creating next-generation FPs on high-fidelity data.

Huang, Xu [University of California, Berkeley, CA ↗
DOE OSTI2024

Uncertainty-refined image segmentation under domain shift

Digital image segmentation is provided. The method comprises training a neural network for image segmentation with a labeled training dataset from a first domain, wherein a subset of nodes in the neural net are dropped out during training. The neural network receives image data from a second, different domain. A vector of N values that sum to 1 is calculated for each image element, wherein each value represents an image segmentation class. A label is assigned to each image element according to the class with the highest value in the vector. Multiple inferences are performed with active dropout layers for each image element, and an uncertainty value is generated for each image element. Uncertainty is resolved according to expected characteristics. The label of any image element with an uncertainty above a threshold is replaced with a new label corresponding to a segmentation class based on domain knowledge.

Martinez, Carianne↗
DOE OSTI2025

A new database website for nuclear level densities

We introduce a new open-access, web-based database (http://nld.ascsn.net), Current Archive of Nuclear Density of Levels (CANDL), that hosts experimental nuclear level density (NLD) datasets from a variety of techniques and energy ranges. Built using the Dash framework in Python, the database is designed to be interactive and user-friendly, allowing researchers to search, visualize, fit, and export NLD data with minimal effort. This resource includes data extracted from evaporation spectra, Oslo method variants, and other experimental techniques that cover excitation energies beyond the neutron resonance region. The database supports on-the-fly fitting with two widely-used phenomenological models—the Constant Temperature (CT) model and the Back-Shifted Fermi Gas (BSFG) model—selected for their simplicity and computational efficiency. Future versions aim to include additional datasets and model types, as well as easy-to-use interfaces to data science techniques. Here, this platform offers a vital tool for the nuclear physics, astrophysics, medicine, and reactor design communities.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Scalable 3D reconstruction for X-ray single particle imaging with online machine learning

X-ray free-electron lasers offer unique capabilities for measuring the structure and dynamics of biomolecules, helping us understand the basic building blocks of life. Notably, high-repetition-rate free-electron lasers enable single particle imaging, where individual, weakly scattering biomolecules are imaged under near-physiological conditions with the opportunity to access fleeting states that cannot be captured in cryogenic or crystallized conditions. Existing X-ray single particle reconstruction algorithms, which estimate the particle orientation for each image independently, are slow and memory-intensive when handling the massive datasets generated by emerging free-electron lasers. Here, we introduce X-RAI (X-Ray single particle imaging with Amortized Inference), an online reconstruction framework that estimates the structure of 3D macromolecules from large X-ray single particle datasets. X-RAI consists of a convolutional encoder, which amortizes pose estimation over large datasets, as well as a physics-based decoder, which employs an implicit neural representation to enable high-quality 3D reconstruction in an end-to-end, self-supervised manner. We demonstrate that X-RAI achieves state-of-the-art performance for small-scale datasets in simulation and challenging experimental settings and demonstrate its unprecedented ability to process large datasets containing millions of diffraction images in an online fashion. These abilities signify a paradigm shift in X-ray single particle imaging towards real-time reconstruction.

Computer science↗
DOE OSTI2026

Frequentist cosmological constraints from full-shape clustering measurements in DESI DR1

We present a frequentist analysis of clustering measurements from Data Release 1 of the Dark Energy Spectroscopic Instrument (DESI) using the standard profile likelihood method. While Bayesian inferences for effective field theory models of galaxy clustering can be highly sensitive to prior choices for extended cosmological models, frequentist inferences are not susceptible to such effects. We compare frequentist and Bayesian constraints for the parameter set {σ 8 , H 0 , Ω m , w 0 , w a } using the full-shape power spectrum multipoles, post-reconstruction baryon acoustic oscillation (BAO) measurements, and external datasets from the CMB and type Ia supernovae measurements. The frequentist confidence intervals are significantly shifted relative to the Bayesian credible intervals for the w 0 w a CDM model, unless supernovae data are included. When DESI full-shape and BAO data are fit jointly, we obtain the following 1σ frequentist confidence intervals for ΛCDM (w 0 w a CDM): σ 8 = 0.863 +0.048 -0.040 , H 0 = 68.96 +0.81 -0.80 km s -1 Mpc -1 , Ω m = 0.3034 ± 0.0110 (σ 8 = 0.782 +0.060 -0.036 , H 0 = 63.7 +4.2 -2.0 km s -1 Mpc -1 , Ω m = 0.378 +0.024 -0.047 , w 0 = -0.16 +0.10 -0.50 , w a = -3.0 +1.7 ), corresponding to 0.8σ, 0.3σ, 0.7σ (2.1σ, 4.1σ, 6.5σ, 6.3σ, 6.6σ) shifts between the maximum likelihood estimate and the Bayesian posterior mean for ΛCDM (w 0 w a CDM) respectively.

Bayesian reasoning↗
DOE OSTI2022

Rapidly changing high-latitude seasonality: Implications for the 21st century carbon cycle in Alaska: Modeling Archive

This Modeling Archive is in support of an NGEE Arctic publication "Rapidly changing high-latitude seasonality: Implications for the 21st century carbon cycle in Alaska" DOI: 10.1088/1748-9326/ac4362.The dataset contains "ecosys" model outputs reported in Shirley et al. (2022) that explored how climate warming will shift carbon cycle seasonality in Alaska. The study used analysis of seasonal dynamics to support the prediction that high-latitude ecosystems are carbon sinks and will continue to accumulate carbon throughout the century. The study predicts surprisingly large increases in spring net carbon uptake that result in larger net carbon uptake in spring than in summer by year 2100. Included as *.nc files are modeled daily net ecosystem exchange (NEE), net primary productivity (NPP), heterotrophic respiration (Rh), autotrophic respiration (Ra), gross primary productivity (GPP), and plant nitrogen uptake. The output is provided (1) at 25 km resolution across Alaska; (2) for years 2010 through 2100. Scripts to generate publication figures included as text files (*.m) from MATLAB. A user guide included as *.pdf.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Self-diagnosis of model suitability for continuous measurements of stream-dissolved organic carbon derived from in situ UV–visible spectroscopy

Application of high-frequency monitoring of dissolved organic carbon (DOC) is difficult in instances where training datasets are challenging to develop (e.g., remote locations) and the relationship between optical features and DOC concentration changes due to environmental or landscape shifts (e.g., climate or land-use change). We developed and compared three partial least squares (PLS) models using in situ water level measurements, conductivity, and UV–Vis spectral attenuation to predict DOC. Two site-specific models were developed using data from a hillslope-dominated forest or a low-relief wetland-pond-dominated stream catchment. The third model, using data from both sites, exhibited the best performance (DOC range = 4–15.5 mg C L −1 , mean = 8.38 mg C L −1 , training RMSE = 0.34 mg C L −1 , internal validation RMSE = 0.50 mg C L −1 , external validation RMSE = 2.43 mg C L −1 ). We further demonstrate using PLS model statistics to monitor performance and elucidate when and how models should be updated. These statistics, Hotelling's T 2 and squared prediction errors, are useful consistency checks for the predictions made and detect underlying inconsistencies that, if undetected, can reduce the robustness of DOC prediction. For example, via the T 2 statistic, we identified the summer–autumn transition as a period when DOC composition differed from what was represented in the training dataset. We also determined that elevated SUVA 254 values contributed to the overall bias observed in predictions made during the subsequent year as part of the external validation. This enabled the application of a bias correction that reduced the RMSE from 2.43 to 0.89 mg C L −1 . The method presented here could be applied to future monitoring programs enabling model updates to monitor DOC fluxes accurately from optical datasets (e.g., attenuance or fluorescence) in the face of developing datasets in remote locations or environmental change. In conclusion, implementation of this approach may also identify possible regime shifts or landscape and hydrologic change associated with climate and other environmental changes relevant to terrestrial to aquatic fluxes.

UV-VIS spectroscopy↗
DOE OSTI2021

Phase-sensitive γ-encoded recoupling of heteronuclear dipolar interactions and 1 H chemical shift anisotropy

γ-encoded recoupling sequences are known to produce strong amplitude modulations that lead to sharp doublets when Fourier transformed. These doublets depend very little on the recoupled tensor asymmetry and thus enable for the straightforward determination of dynamic order parameters. It can, however, be difficult to measure small anisotropies, or small order parameters, using such sequences; the resonances from the doublet may overlap with each other, or with the zero-frequency glitch. This limitation has prevented the widespread use of 1 H chemical shift anisotropy (CSA) for the measurement of dynamics, particularly for CH protons which typically have CSAs of only a few ppm when immobile. In this work, we introduce a simple modification to the traditional 1 H CSA and proton-detected local field pulse sequences that enables the acquisition of a hypercomplex dataset and the removal of the uncorrelated magnetization that results in the zero-frequency glitch. These new sequences then yield a frequency shift in the indirect dimension, rather than a splitting, which is easily identifiable even in cases of weak interactions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

Data and scripts associated with a manuscript analyzing ELM-FATES parameter sensitivity under pre-fire and postfire scenarios using machine learning

NOTE: The manuscript associated with this data package is currently in review. The data may be revised based on reviewer feedback. Upon manuscript acceptance, this data package will be updated with the final dataset and additional metadata. This data package is associated with the manuscript “Fire Severity-Dependent Shifts in Vegetation Parameter Sensitivity: A Pre- and Post-Fire Analysis Using ELM-FATES and Explainable AI” submitted to Journal of Advances in Modeling Earth Systems (Zahura et al. 2026). The study examines vegetation physiological parameters controlling pre-fire and post-fire vegetation dynamics. To support this analysis, 73 vegetation parameters in Functionally Assembled Terrestrial Ecosystem Simulator (FATES) (Fisher et al., 2018) , which is coupled with E3SM (Energy Exascale Earth System Model) land model (ELM, ELM-FATES), were perturbed using a Sobol sequence to generate 1,024 ensemble members for two plant functional types: needleleaf evergreen extratropical trees (NEET) and C3 grass. Simulations were conducted for the pre-fire period (2016) and post-fire period (2018–2023). Burn severity was represented by modifying the Nesterov index in FATES to 75,000, 150,000, and 300,000 for low, moderate, and high severity, respectively. A no-fire scenario was also included. Simulations were performed for 16 grid cells in the American River Watershed across different burn severities and plant functional types. XGBoost (eXtreme Gradient Boosting) models were trained using the parameter ensembles and ELM-FATES-simulated outputs, including leaf area index (LAI), gross primary productivity (GPP), aboveground biomass, vegetation evaporation, transpiration, and soil evaporation. Models were trained separately for each year and burn severity, followed by SHAP (SHapley Additive exPlanations) analysis to identify changes in dominant parameters after fire disturbance. For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. The data package contains the ELM-FATES simulation data. The scripts and data related to the analysis will be added later. The inputs and outputs from ELM-FATES are inside the “FATES” folder. “FATES_domain_surface” contains the domain and surface netcdfs that were used to run ELM-FATES in the study area. “FATES_parameters” contains the 1024 ensembles that were generated using Sobol sequence. “FATES_outputs” folder contains ELM-FATES simulated variables. All files are .csv and .nc (NetCDF).

Aboveground biomass↗
DOE OSTI2022

Uncertainty-refined image segmentation under domain shift

A method for digital image segmentation is provided. The method comprises training a neural network for image segmentation with a labeled training dataset from a first domain, wherein a subset of nodes in the neural net are dropped out during training. The neural network receives image data from a second, different domain. A vector of N values that sum to 1 is calculated for each image element, wherein each value represents an image segmentation class. A label is assigned to each image element according to the class with the highest value in the vector. Multiple inferences are performed with active dropout layers for each image element, and an uncertainty value is generated for each image element. The label of any image element with an uncertainty value above a predefined threshold is replaced with a new label corresponding to the class with the next highest value.

Martinez, Carianne↗
DOE OSTI2024

Solar, Wind, and Load Forecasting Dataset for MISO, NYISO, and SPP Balancing Areas

The Performance-based Energy Resource Feedback, Optimization, and Risk Management (PERFORM) program is an initiative intended to foster "a fundamental shift in grid management rooted in an understanding of asset risk and system risk" (ARPA-E 2020). Launched by the Advanced Research Projects Agency-Energy (ARPA-E), the program supports efforts to incorporate uncertainty in electric power decision making. In support of PERFORM, the National Renewable Energy Laboratory (NREL) has produced a set of time-coincident forecasts of solar, wind, and load profiles. As part of Phase I of the PERFORM effort, NREL created a dataset that consists of one year of time-coincident load, wind, and solar actuals and probabilistic forecasts based on data from the Electric Reliability Council of Texas (ERCOT) (Bryce et al. 2023). In Phase II, NREL developed similar datasets for three other U.S. Independent System Operators (ISO): the Midcontinent Independent System Operator (MISO), the New York Independent System Operator (NYISO), and the Southwest Power Pool (SPP).

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Time-temperature-superposition analysis of diverse datasets by the minimum-arclength method: long-term prediction with uncertainty margins

In a recent publication, we carried out an extensive analysis of an unsupervised method of determining optimum shift factors in time-temperature-superposition of accelerated-aging data that involves minimizing the vertical arclength to obtain the master curve. For synthetic Arrhenius data with a variety of noise distributions, the work showed that, in conjunction with bootstrap-resampling, the method can produce reliable estimates of the mean activation energy along with uncertainty quantification. Here, we apply the above method to six different datasets taken from the published literature and demonstrate accurate prediction of mean activation energy from the data as-is without the need for any pre-processing or fitting. We also compare uncertainty margins computed by second-order bootstrap with that by linear regression theory and show that the former appears to provide consistent margins in the presence of common noise types in real data, including intra-isotherm measurement-errors, sample-to-sample variations, and intrinsic deviation from perfect Arrhenius behavior.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Resource-Adaptive Federated Text Generation with Differential Privacy

In cross-silo federated learning (FL), sensitive text datasets remain confined to local organizations due to privacy regulations, making repeated training for each downstream task both communication-intensive and privacy-demanding. A promising alternative is to generate differentially private (DP) synthetic datasets that approximate the global distribution and can be reused across tasks. However, pretrained large language models (LLMs) often fail under domain shift, and federated finetuning is hindered by computational heterogeneity: only resource-rich clients can update the model, while weaker clients are excluded, amplifying data skew and the adverse effects of DP noise. We propose a flexible participation framework that adapts to client capacities. Strong clients perform DP federated finetuning, while weak clients contribute through a lightweight DP voting mechanism that refines synthetic text. To ensure the synthetic data mirrors the global dataset, we apply control codes (e.g., labels, topics, metadata) that represent each client’s data proportions and constrain voting to semantically coherent subsets. This two-phase approach requires only a single round of communication for weak clients and integrates contributions from all participants. Experiments show that our framework improves distribution alignment and downstream robustness under DP and heterogeneity.

Wang, Jiayi [ORNL]↗
DOE OSTI2021

Land-use harmonization datasets for annual global carbon budgets

Abstract. Land-use change has been the dominant source of anthropogenic carbon emissions for most of the historical period and is currently one of the largest and most uncertain components of the global carbon cycle. Advancing the scientific understanding on this topic requires that the best data be used as input to state-of-the-art models in well-organized scientific assessments. The Land-Use Harmonization 2 dataset (LUH2), previously developed and used as input for simulations of the 6th Coupled Model Intercomparison Project (CMIP6), has been updated annually to provide required input to land models in the annual Global Carbon Budget (GCB) assessments. Here we discuss the methodology for producing these annual LUH2-GCB updates and extensions which incorporate annual wood harvest data updates from the Food and Agriculture Organization (FAO) of the United Nations for dataset years after 2015 and the History Database of the Global Environment (HYDE) gridded cropland and grazing area data updates (based on annual FAO cropland and grazing area data updates) for dataset years after 2012, along with extrapolations to the current year due to a lag of 1 or more years in the FAO data releases. The resulting updated LUH2-GCB datasets have provided global, annual gridded land-use and land-use-change data relating to agricultural expansion, deforestation, wood harvesting, shifting cultivation, regrowth and afforestation, crop rotations, and pasture management and are used by both bookkeeping models and dynamic global vegetation models (DGVMs) for the GCB. For GCB 2019, a more significant update to LUH2 was produced, LUH2-GCB2019 (https://doi.org/10.3334/ORNLDAAC/1851, Chini et al., 2020b), to take advantage of new data inputs that corrected cropland and grazing areas in the globally important region of Brazil as far back as 1950. From 1951 to 2012 the LUH2-GCB2019 dataset begins to diverge from the version of LUH2 used for the World Climate Research Programme's CMIP6, with peak differences in Brazil in the year 2000 for grazing land (difference of 100 000 km2) and in the year 2009 for cropland (difference of 77 000 km2), along with significant sub-national reorganization of agricultural land-use patterns within Brazil. The LUH2-GCB2019 dataset provides the base for future LUH2-GCB updates, including the recent LUH2-GCB2020 dataset, and presents a starting point for operationalizing the creation of these datasets to reduce time lags due to the multiple input dataset and model latencies.

Chini, Louise (ORCID:0000000290703505)↗
DOE OSTI2025

Bridging the Gap Between Astronomical Datasets: From Proof-of-Concept to AI Model Deployment with Domain Adaptation

Artificial Intelligence is transforming astrophysics, from studying stars and galaxies to analyzing cosmic large-scale structures. However, a critical challenge arises when AI models trained on simulations or past observational data are applied to new observation— leading to domain shifts, reduced robustness, and increased uncertainty of model predictions. This talk will explore these issues, highlighting examples such as galaxy morphology classification and cosmological parameter inference, where AI struggles to adapt across different datasets. We will discuss domain adaptation as a strategy to improve model generalization and mitigate biases—essential for making AI-driven discoveries reliable. Notably, these challenges extend beyond astrophysics, affecting AI applications across physics and other scientific domains. Addressing them is essential for maximizing AI’s impact in advancing scientific research.

Ćiprijanović, Aleksandra [Fermilab]↗
DOE OSTI2023

Datasets used in manuscript, 'High-Throughput Chromosomal Confirmation Capture (Hi-C) Metagenome Sequencing Reveals Moisture Impact on Soil Phage-Host Interactions'

Soil moisture shifts have largely unknown impacts on soil virus-host interactions. Here, we applied high-throughput chromosomal confirmation capture (Hi-C) metagenomics to link phage with their hosts in soils under wet and dry conditions. Bulk metagenomes and metatranscriptomes were analyzed from the same soil incubations. Host-associated phage diversity and the number of viruses per host increased following soil desiccation. Under wet conditions, the viral enrichment and host abundances were significantly negatively correlated, but the transcriptional activities of the phage were higher. Together, these results suggest that there was a general transition of phage from lytic to lysogenic during drying and that viral infection was higher under dry conditions. Soil desiccation also caused shifts in phage hosts and some of these were central in microbial co-occurrence networks, highlighting the impact of soil phages on microbiome structure. This study provides the first empirical evidence of phage-mediated bacterial dynamics during soil desiccation.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Class imbalance in out-of-distribution datasets: Improving the robustness of the TextCNN for the classification of rare cancer types

In the last decade, the widespread adoption of electronic health record documentation has created huge opportunities for information mining. Natural language processing (NLP) techniques using machine and deep learning are becoming increasingly widespread for information extraction tasks from unstructured clinical notes. Disparities in performance when deploying machine learning models in the real world have recently received considerable attention. In the clinical NLP domain, the robustness of convolutional neural networks (CNNs) for classifying cancer pathology reports under natural distribution shifts remains understudied. In this research, we aim to quantify and improve the performance of the CNN for text classification on out-of-distribution (OOD) datasets resulting from the natural evolution of clinical text in pathology reports. We identified class imbalance due to different prevalence of cancer types as one of the sources of performance drop and analyzed the impact of previous methods for addressing class imbalance when deploying models in real-world domains. Our results show that our novel class-specialized ensemble technique outperforms other methods for the classification of rare cancer types in terms of macro F1 scores. We also found that traditional ensemble methods perform better in top classes, leading to higher micro F1 scores. Based on our findings, we formulate a series of recommendations for other ML practitioners on how to build robust models with extremely imbalanced datasets in biomedical NLP applications.

60 APPLIED LIFE SCIENCES↗