SEARCH · Search NASA

Results for “dataset shift”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Using Long-Short Term Memory Models to Predict Solar Active Regions Emergence

We train Long Short-Term Memory (LSTM) models that predict the formation of active regions (ARs), the main source of eruptive solar activity. Using the Doppler shift velocity, the continuum intensity and the magnetic field full-disk maps from SDO/HMI we have created time-series datasets of acoustic power and magnetic flux which are used to train Long Short-Term Memory (LSTM) models on predicting decreases in continuum intensity 12 hours in advance. Testing of the models' performance was done on data from 5 ARs, unseen from the model during training. The model predicted the emergence of AR11726, AR13165 and AR13179, 10, 29 and 5 hours in advance, and variations of this model achieved average RMSE values of 0.11 for both active and quiet parts of the solar disc, showing the ability of the model to capture acoustic power anomalies and predict continuum intensity variations. This work sets the foundations for the very first ML-aided prediction of solar ARs.

SMD↗

DOE OSTI2025

The Importance of Being Adaptable: An Exploration of the Power and Limitations of Domain Adaptation for Simulation-Based Inference with Galaxy Clusters

The application of deep machine learning methods in astronomy has exploded in the last decade, with new models showing remarkably improved performance on benchmark tasks. Not nearly enough attention is given to understanding the models' robustness, especially when the test data are systematically different from the training data, or "out of domain." Domain shift poses a significant challenge for simulation-based inference, where models are trained on simulated data but applied to real observational data. In this paper, we explore domain shift and test domain adaptation methods for a specific scientific case: simulation-based inference for estimating galaxy cluster masses from X-ray profiles. We build datasets to mimic simulation-based inference: a training set from the Magneticum simulation, a scatter-augmented training set to capture uncertainties in scaling relations, and a test set derived from the IllustrisTNG simulation. We demonstrate that the Test Set is out of domain in subtle ways that would be difficult to detect without careful analysis. We apply three deep learning methods: a standard neural network (NN), a neural network trained on the scatter-augmented input catalogs, and a Deep Reconstruction-Regression Network (DRRN), a semi-supervised deep model engineered to address domain shift. Although the NN improves results by 17% in the Training Data, it performs 40% worse on the out-of-domain Test Set. Surprisingly, the Scatter-Augmented Neural Network (SANN) performs similarly. While the DRRN is successful in mapping the training and Test Data onto the same latent space, it consistently underperforms compared to a straightforward Yx scaling relation. These results serve as a warning that simulation-based inference must be handled with extreme care, as subtle differences between training simulations and observational data can lead to unforeseen biases creeping into the results.

Ntampaka, Michelle [Baltimore, Space Telescope Sci↗

DOE OSTI2023

Low-Temperature Geothermal Geospatial Datasets: An Example from Alaska

This project is a component of a broader effort focused on geothermal heating and cooling (GHC) with the aim of illustrating the numerous benefits of incorporating GHC and geothermal heat exchange (GHX) into community energy planning and national decarbonization strategies. To better assist private sector investment, it is currently necessary to define and assess the potential of low-temperature geothermal resources. For shallow GHC/GHX fields, there is no formal compilation of subsurface characteristics shared among industry practitioners that can improve system design and operations. Alaska is specifically noted in this work, because heretofore, it has not received a similar focus in geothermal potential evaluations as the contiguous United States. The methodology consists of leveraging relevant data to generate a baseline geospatial dataset of low-temperature resources (less than 150 degrees C) to compare and analyze information accessible to anyone trying to understand the potential of GHC/GHX and small-scale low-temperature geothermal power in Alaska (e.g., energy modelers, communities, planners, and policymakers). Importantly, this project identifies data related to (1) the evaluation of GHC/GHX in the shallow subsurface, and (2) the evaluation of low-temperature geothermal resource availability. Additionally, data is being compiled to assess repurposing of oil and gas wells to contribute co-produced fluids toward the geothermal direct use and heating and cooling resource potential. In this work we identified new data from three different datasets of isolated geothermal systems in Alaska and bottom-hole temperature data from oil and gas wells that can be leveraged for evaluation of low-temperature geothermal resource potential. The goal of this project is to facilitate future deployment of GHC/GHX analysis and community-led programs and update the low-temperature geothermal resources assessment of Alaska. A better understanding of shallow potential for GHX will improve design and operations of highly efficient GHC systems. The deployment and impact that can be achieved for low-temperature geothermal resources will contribute to decarbonization goals and facilitate widespread electrification by shaving and shifting grid loads.

15 GEOTHERMAL ENERGY↗

NASA NTRS2012

The Effects of Climate Variability on Phytoplankton Composition in the Equatorial Pacific Ocean using a Model and a Satellite-Derived Approach

Compared the interannual variation in diatoms, cyanobacteria, coccolithophores and chlorophytes from the NASA Ocean Biogeochemical Model with those derived from satellite data (Hirata et al. 2011) between 1998 and 2006 in the Equatorial Pacific. Using NOBM, La Ni a events were characterized by an increase in diatoms (correlation with MEI, r=-0.81, P<0.05), while cyanobacteria concentrations decreased significantly (r=0.61; P<0.05). El Nino produced the reverse pattern, with cyanobacteria populations increasing while diatoms plummeted. This represented a radical shift in the phytoplankton community in response to climate variability. However, satellite-derived phytoplankton groups were all negatively correlated with climate variability (r ranged from -0.39 for diatoms to -0.64 for coccolithophores, P<0.05). Spatially, the satellite-derived approach was closer to an independent in situ dataset for all phytoplankton groups except diatoms than NOBM. However, the different responses of phytoplankton to intense interannual events in the Equatorial Pacific raises questions about the representation of phytoplankton dynamics in models and algorithms: is a phytoplankton community shift as in the model or an across-the-board change in abundances of all phytoplankton as in the satellite-derived approach.

Rousseaux, C. S.↗

DOE OSTI2023

KBase Credit Metadata Schema

As part of KBase’s commitment to promote open science, we offer users the ability to obtain a DOI (Digital Object Identifier) for their work, which can then be cited in an associated science publication. To further support the community-wide shift towards FAIR (Findable, Accessible, Interoperable, Reusable) data, KBase is expanding our data descriptors so that KBase DOIs have comprehensive citations for datasets, in addition to referencing publications or software used in the workflow. This helps encourage a culture of giving attribution for all research inputs and outputs; standard practice for literature, but still relatively new for software products or datasets. It also promotes open science by building trust that contributors get credit for their work, and accelerates knowledge discovery by supporting and incentivizing the release of data.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2023

Slater transition methods for core-level electron binding energies

Methods for computing core-level ionization energies using self-consistent field (SCF) calculations are evaluated and benchmarked. These include a “full core hole” (or “ΔSCF”) approach that fully accounts for orbital relaxation upon ionization, but also methods based on Slater’s transition concept in which the binding energy is estimated from an orbital energy level that is obtained from a fractional-occupancy SCF calculation. A generalization that uses two different fractional-occupancy SCF calculations is also considered. The best of the Slater-type methods afford mean errors of 0.3–0.4 eV with respect to experiment for a dataset of K-shell ionization energies, a level of accuracy that is competitive with more expensive many-body techniques. An empirical shifting procedure with one adjustable parameter reduces the average error below 0.2 eV. Here, this shifted Slater transition method is a simple and practical way to compute core-level binding energies using only initial-state Kohn–Sham eigenvalues. It requires no more computational effort than ΔSCF and may be especially useful for simulating transient x-ray experiments where core-level spectroscopy is used to probe an excited electronic state, for which the ΔSCF approach requires a tedious state-by-state calculation of the spectrum. As an example, we use Slater-type methods to model x-ray emission spectroscopy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2020

How analytic choices can affect the extraction of electromagnetic form factors from elastic electron scattering cross section data

Scientists often try to incorporate prior knowledge into their regression algorithms, such as a particular analytic behavior or a known value at a kinematic endpoint. Unfortunately, there is often no unique way to make use of this prior knowledge, and thus, different analytic choices can lead to very different regression results from the same set of data. In this paper, to illustrate this point in the context of the proton electromagnetic form factors, we use the Mainz elastic data with its 1422 cross section points and 31 normalization parameters. Starting with a complex unbound non-linear regression, we will show how the addition of a single theory-motivated constraint removes an oscillation from the magnetic form factor and shifts the extracted proton charge radius. We then repeat both regressions using the same algorithm, but with a rebinned version of the Mainz dataset. These examples illustrate how analytic choices, such as the function that is being used or even the binning of the data, can dramatically affect the results of a complex regression. These results also demonstrate why it is critical when using regression algorithms to have either a physical model in mind or a firm mathematical basis

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2020

Apparatus and method for determining state of change (SOC) and state of health (SOH) of electrical cells

Systems and methods for prediction of state of charge (SOH), state of health (SOC) and other characteristics of batteries using acoustic signals, includes determining acoustic data at two or more states of charge and determining a reduced acoustic data set representative of the acoustic data at the two or more states of charge. The reduced acoustic data set includes time of flight (TOF) shift, total signal amplitude, or other data points related to the states of charge. Machine learning models use at least the reduced acoustic dataset in conjunction with non-acoustic data such as voltage and temperature for predicting the characteristics of any other independent battery.

25 ENERGY STORAGE↗

DOE OSTI2022

Data Curation for Machine Learning Applied to Geothermal Power Plant Operational Data for GOOML: Geothermal Operational Optimization with Machine Learning: Preprint

Geothermal Operational Optimization with Machine Learning (GOOML) is a transferable and extensible component-based geothermal asset modeling framework that considers complex steamfield relationships and identifies optimization prospects using a data-driven approach to physics-guided, data-centric machine learning. This framework has been used to develop digital twins that provide steamfield operators with operational environments to analyze and understand historical and forecasted power production, explore new steamfield configuration possibilities, and seek optimal asset management in real world applications. To create, test, and apply the GOOML framework, diverse time-series datasets spanning multiple years were sourced from various geothermal power plant components within several complex real-world geothermal operations. These operations are based in the United States and New Zealand and include a variety of technologies, end-uses and configurations, collectively covering nearly all relevant operating conditions for modern geothermal fields. Datasets were acquired from multiple sources to ensure that machine learning experiments generalized properly to various operating conditions. It was found that the data varied in quality, format, and completeness. To ensure consistency between the various datasets, a standardized data curation process was developed to reliably streamline data preparation. This paper will discuss best practices as learned from the GOOML data curation process which takes the following steps: 1) acquisition of large quantities of data from power plant operators, 2) digestion of data to gain an initial understanding of what is included, 3) data transformation, which includes converting the data into a standardized machine-readable format so that they can be visualized, quality checked, and cleaned, 4) quality assurance and quality control, involving identification of significant data gaps and apparent anomalies through mapping of data features to real world componentry via the GOOML historical model, followed by discussion with modelers and power plant operators to identify additional data needs and to resolve issues, 5) use in machine learning algorithms, and 6) repetition of steps one through five until all data needs are met and data are deemed suitable for producing trustworthy modeling results which may be disseminated, ideally along with the curated dataset. This iterative process is focused on improving the quality of the data rather than tuning machine learning model parameters and supports a shift towards data-centric AI as a means to improving real-world applicability of geothermal machine learning projects.

access↗

DOE OSTI2023

Apparatus and method for determining state of change (SOC) and state of health (SOH) of electrical cells

Steingart, Daniel Artemis↗

DOE OSTI2024

An overview of the Western United States Dynamically Downscaled Dataset (WUS-D3)

Abstract. Predicting future climate change over a region of complex terrain, such as the western United States (US), remains challenging due to the low resolution of global climate models (GCMs). Yet the climate extremes of recent years in this region, such as floods, wildfires, and drought, are likely to intensify further as climate warms, underscoring the need for high-quality and high-resolution predictions. Here, we present an ensemble of dynamically downscaled simulations over the western US from 1980–2100 at 9 km grid spacing, driven by 16 latest-generation GCMs. This dataset is titled the Western US Dynamically Downscaled Dataset (WUS-D3). We describe the challenges of producing WUS-D3, including GCM selection and technical issues, and we evaluate the simulations' realism by comparing historical results to temperature and precipitation observations. The future downscaled climate change signals are shaped in physically credible ways by the regional model's more realistic coastlines and topography. (1) The mean warming signals are heavily influenced by more realistic snowpack. (2) Mean precipitation changes are often consistent with wetting on the windward side of mountain complexes, as warmer, moister air masses are uplifted orographically during precipitation events. (3) There are large fractional precipitation increases on the lee side of mountain complexes, leading to potentially significant changes in water resources and ecology in these arid landscapes. (4) Increases in precipitation extremes are generally larger than in the GCMs, driven by locally intensified atmospheric updrafts tied to sharper, more realistic gradients in topography. (5) Changes in temperature extremes are different from what is expected by a shift in mean temperature and are shaped by local atmospheric dynamics and land surface feedbacks. Because of its high resolution, comprehensiveness, and representation of relevant physical processes, this dataset presents a unique opportunity to evaluate societally relevant future changes in western US climate.

Rahimi, Stefan (ORCID:0000000331884462)↗

DOE OSTI2021

SFA-VirOmics

PNNL's Soil Microbiome Science Focus Area (SFA) is focused on understanding the basic biology underpinning how interactions among various soil microbial community members, across trophic levels, lead to the emergence of community functions. Moisture, in particular, drives microbial interactions and influences everything from cell function to substrate fate within soils. The group predicts this results in repeatable, predictable phenotypes. The sum of these phenotypes comprises the “soil metaphenome”. Understanding how the soil metaphenome shifts in response to moisture will provide a basis for modeling and predicting these shifts in reaction network responses. Visit the PNNL Soil Microbiome Science Focus Area Program homepage for more information. PNNL’s Soil Microbiome SFA Virome Dataset Annotation page is an extension to the PNNL Soil Microbiome SFA repository on DataHub allows for exploring and downloading integrated experimental omics dataset annotations, associated experimental metadata, pre- and post-processed viral data files, and other associated materials directly related to experimental viral project data.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

Enhancing synchrotron radiation micro-CT images using deep learning: an application of Noise2Inverse on bone imaging

In bone-imaging research, in situ synchrotron radiation micro-computed tomography (SRµCT) mechanical tests are used to investigate the mechanical properties of bone in relation to its microstructure. Low-dose computed tomography (CT) is used to preserve bone's mechanical properties from radiation damage, though it increases noise. To reduce this noise, the self-supervised deep learning method Noise2Inverse was used on low-dose SRµCT images where segmentation using traditional thresholding techniques was not possible. Simulated-dose datasets were created by sampling projection data at full, one-half, one-third, one-fourth and one-sixth frequencies of an in situ SRµCT mechanical test. After convolutional neural networks were trained, Noise2Inverse performance on all dose simulations was assessed visually and by analyzing bone microstructural features. Visually, high image quality was recovered for each simulated dose. Lacunae volume, lacunae aspect ratio and mineralization distributions shifted slightly in full, one-half and one-third dose network results, but were distorted in one-fourth and one-sixth dose network results. Following this, new models were trained using a larger dataset to determine differences between full dose and one-third dose simulations. Significant changes were found for all parameters of bone microstructure, indicating that a separate validation scan may be necessary to apply this technique for microstructure quantification. Noise present during data acquisition from the testing setup was determined to be the primary source of concern for Noise2Inverse viability. While these limitations exist, incorporating dose calculations and optimal imaging parameters enables self-supervised deep learning methods such as Noise2Inverse to be integrated into existing experiments to decrease radiation dose.

Obata, Yoshihiro (ORCID:0000000303659129)↗

DOE OSTI2026

Alcock-Paczynski Blinding Scheme for the Ly-$α$ Forest Analysis

We present and validate a blinding method for the Lyman-$α$ (Ly$α$) forest analysis based on a modification of the Alcock-Paczynski test. In order to hide the background expansion history, the method employs a geometrical shift of each quasar (QSO) forest in wavelength space, once the quasar continuum has been fitted and the fluctuation field is extracted. The redshift positions for the QSO sample are also changed in a consistent manner. We show that the method remains effective when applied to real data, where contamination from metals and Lyman-$β$ is intrinsically mixed with the Lyman-$α$ forest. This limitation is primarily visible in the 1D correlation function, where other blinding strategies can mitigate the effect. To assess its effectiveness, the prescription is tested against a series of datasets of increasing complexity: from idealized low-noise mocks, to realistic DESI year one synthetic datasets, and finally to data from DESI first data release (DR1), using both the auto (Ly$α\times$Ly$α$) and cross (Ly$α\times$ QSO) correlations. We find that the method robustly shifts the BAO peak position from the 3D correlation functions to the expected value for cosmology changes of around 5% in the matter content, without altering the shape of the posteriors in the model parameters. In conclusion, this catalog-level blinding strategy is a viable method for cosmological inference with the Lyman-$α$ forest, particularly if a cross-analysis with other tracers using the same blinding strategy is pursued.

Perez-Sanchez, G. [Guanajuato U.] (ORCID:000900096↗

DOE OSTI2026

SPRUCE FT-ICR MS, Bulk Chemistry, and Mass Loss from Litter Decomposition Study in Experimental Plots, Marcell Experimental Forest, Minnesota, 2015-2017

This dataset contains molecular, bulk chemical, and mass loss measurements from a litter decomposition study at the Spruce and Peatland Responses Under Changing Environments (SPRUCE) experimental site within the Marcell Experimental Forest in northern Minnesota, USA. This site is in a Sphagnum spp. ombrotrophic bog forest. Litterbags were deployed into the peat in September 2015 across three warming levels (+0, +4.5, and +9°C) under ambient and elevated carbon dioxide (CO₂ - +500 ppm) and retrieved after roughly 0.5, 1, and 2 years of field incubation (2015-09-23 to 2017-08-02). Litterbags containing six peatland litter types: black spruce needles (Picea mariana - SPL), spruce fine roots (SPR), Sphagnum angustifolium (ANG), Sphagnum magellanicum (MAG), Labrador tea leaves (Rhododendron groenlandicum - LTL), and Labrador tea roots (LTR). Molecular composition of water-soluble organic matter extracts was characterized using Fourier Transform Ion Cyclotron Resonance Mass Spectrometry (FT-ICR MS) at 9.4 Tesla, operated in negative ion mode with electrospray ionization, providing molecular formula assignments and compound-class distributions across the decomposition time series. Bulk chemical characterization included elemental analysis (percent carbon, nitrogen, and phosphorus) and Fourier Transform Infrared Spectroscopy (FTIR) to quantify functional group composition. Litter mass loss was tracked gravimetrically at each retrieval interval, expressed as percent mass remaining relative to initial dry mass for each litter type and treatment combination. These data are valuable for understanding how vegetation shifts driven by increased atmospheric CO2 and temperature in peatlands alter litter inputs and organic matter stabilization trajectories, with implications for projecting and modeling peatland carbon cycling. This dataset contains two data files in comma-separated value (.csv) format. Additional metadata are provided: two data dictionaries and a file-level metadata file in comma separate (.csv) format and a user guide in PDF (*.pdf) format.

decomposition↗

DOE OSTI2025

Demographic drivers of gut microbiome diversity

Abstract The gut microbiome plays a central role in orchestrating metabolic, immune, and neurological functions essential for human health. While extensive research has explored the effects of diseases and pathological conditions on gut microbiome composition, the influence of demographic factors remains underexplored, limiting our understanding of microbiome variations in disease states. This study addresses this gap by investigating the impact of demographic variables, including age, sex, and geography, on gut microbiome diversity in healthy individuals. Using the American Gut Project’s extensive dataset and the QIIME2 bioinformatics pipeline, we conducted a comprehensive analysis of microbial profiles across diverse demographic groups. Our results revealed significant age-related shifts in microbial richness and composition, and geographic location strongly influenced phylogenetic diversity. In contrast, sex exhibited limited impact on microbial diversity within healthy BMI ranges. These findings highlight the critical role of demographic factors in shaping gut microbiome diversity, providing a foundational framework to better contextualize disease-related microbiome variations and advance personalized healthcare approaches.

Biotechnology & Applied Microbiology↗

DOE OSTI2024

Multiscale Flow for robust and optimal cosmological analysis

We propose Multiscale Flow, a generative Normalizing Flow that creates samples and models the field-level likelihood of two-dimensional cosmological data such as weak lensing. Multiscale Flow uses hierarchical decomposition of cosmological fields via a wavelet basis and then models different wavelet components separately as Normalizing Flows. The log-likelihood of the original cosmological field can be recovered by summing over the log-likelihood of each wavelet term. This decomposition allows us to separate the information from different scales and identify distribution shifts in the data such as unknown scale-dependent systematics. The resulting likelihood analysis can not only identify these types of systematics, but can also be made optimal, in the sense that the Multiscale Flow can learn the full likelihood at the field without any dimensionality reduction. We apply Multiscale Flow to weak lensing mock datasets for cosmological inference and show that it significantly outperforms traditional summary statistics such as power spectrum and peak counts, as well as machine learning–based summary statistics such as scattering transform and convolutional neural networks. We further show that Multiscale Flow is able to identify distribution shifts not in the training data such as baryonic effects. Finally, we demonstrate that Multiscale Flow can be used to generate realistic samples of weak lensing data.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

The future low-temperature geochemical data-scape as envisioned by the U.S. geochemical community

Data sharing benefits the researcher, the scientific community, and the public by allowing the impact of data to be generalized beyond one project and by making science more transparent. However, many scientific communities have not developed protocols or standards for publishing, citing, and versioning datasets. One community that lags in data management is that of low-temperature geochemistry (LTG). This paper resulted from an initiative from 2018 through 2020 to convene LTG and data scientists in the U.S. to strategize future management of LTG data. Through webinars, a workshop, a preprint, a townhall, and a community survey, the group of U.S. scientists discussed the landscape of data management for LTG – the data-scape. Currently this data-scape includes a “street bazaar” of data repositories. This was deemed appropriate in the same way that LTG scientists publish articles in many journals. The variety of data repositories and journals reflect that LTG scientists target many different scientific questions, produce data with extremely different structures and volumes, and utilize copious and complex metadata. Nonetheless, the group agreed that publication of LTG science must be accompanied by sharing of data in publicly accessible repositories, and, for sample-based data, registration of samples with globally unique persistent identifiers. LTG scientists should use certified data repositories that are either highly structured databases designed for specialized types of data, or unstructured generalized data systems. Recognizing the need for tools to enable search and cross-referencing across the proliferating data repositories, the group proposed that the overall data informatics paradigm in LTG should shift from “build data repository, data will come” to “publish data online, cybertools will find”. Funding agencies could also provide portals for LTG scientists to register funded projects and datasets, and forge approaches that cross national boundaries. Finally, the needed transformation of the LTG data culture requires emphasis in student education on science and management of data.

58 GEOSCIENCES↗