Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

Robust historical evapotranspiration trends across climate regimes

Evapotranspiration (ET) links the hydrological, energy and carbon cycles on the land surface. Quantifying ET and its spatio-temporal changes is also key to understanding climate extremes such as droughts, heatwaves and flooding. Regional ET estimates require reliable observation-based gridded ET datasets, and while many have been developed using physically based, empirically based and hybrid techniques, their efficacy, and particularly the efficacy of their uncertainty estimates, is difficult to verify. In this work, we extend the methodology used in Hobeichi et al. (2018) to derive two new versions of the Derived Optimal Linear Combination Evapotranspiration (DOLCE) product, with observationally constrained spatio-temporally varying uncertainty estimates, higher spatial resolution, more constituent products and extended temporal coverage (1980–2018). After demonstrating the efficacy of these uncertainty estimates with out-of-sample testing, we derive novel ET climatology clusters for the land surface, based on the magnitude and variability of ET at each location on land. The new clusters include three wet and three dry regimes and provide an approximation of Köppen–Geiger climate classes. The verified uncertainty estimates and extended time period then allow us to examine the robustness of historical trends spatially and in each of these six ET climatology clusters. We find that despite robust decreasing ET trends in some regions these do not correlate with behavioural ET clusters. Each cluster, and the majority of the Earth's surface, shows clear robust increases in ET over the recent historical period. The new datasets DOLCE V2.1 and DOLCE V3 can be used for benchmarking global ET estimates and for examining ET trends respectively.

54 ENVIRONMENTAL SCIENCES↗

Efficient Active Learning for Gaussian Process Classification by Error Reduction

Active learning sequentially selects the best instance for labeling by optimizing an acquisition function to enhance data/label efficiency. The selection can be either from a discrete instance set (pool-based scenario) or a continuous instance space (query synthesis scenario). In this work, we study both active learning scenarios for Gaussian Process Classification (GPC). The existing active learning strategies that maximize the Estimated Error Reduction (EER) aim at reducing the classification error after training with the new acquired instance in a onestep-look-ahead manner. The computation of EER-based acquisition functions is typically prohibitive as it requires retraining the GPC with every new query. Moreover, as the EER is not smooth, it can not be combined with gradient-based optimization techniques to efficiently explore the continuous instance space for query synthesis. To overcome these critical limitations, we develop computationally efficient algorithms for EER-based active learning with GPC. Further, we derive the joint predictive distribution of label pairs as a one-dimensional integral, as a result of which the computation of the acquisition function avoids retraining the GPC for each query, remarkably reducing the computational overhead. We also derive the gradient chain rule to efficiently calculate the gradient of the acquisition function, which leads to the first query synthesis active learning algorithm implementing EER-based strategies. Our experiments clearly demonstrate the computational efficiency of the proposed algorithms. We also benchmark our algorithms on both synthetic and real-world datasets, which show superior performance in terms of sampling efficiency compared to the existing state-of-the-art algorithms.

97 MATHEMATICS AND COMPUTING↗

Heterogeneous Multi-Domain Dataset Synthesis to Facilitate Privacy and Risk Assessments in Smart City IoT

The emergence of the Smart Cities paradigm and the rapid expansion and integration of Internet of Things (IoT) technologies within this context have created unprecedented opportunities for high-resolution behavioral analytics, urban optimization, and context-aware services. However, this same proliferation intensifies privacy risks, particularly those arising from cross-modal data linkage across heterogeneous sensing platforms. To address these challenges, this paper introduces a comprehensive, statistically grounded framework for generating synthetic, multimodal IoT datasets tailored to Smart City research. The framework produces behaviorally plausible synthetic data suitable for preliminary privacy risk assessment and as a benchmark for future re-identification studies, as well as for evaluating algorithms in mobility modeling, urban informatics, and privacy-enhancing technologies. As part of our approach, we formalize probabilistic methods for synthesizing three heterogeneous and operationally relevant data streams—cellular mobility traces, payment terminal transaction logs, and Smart Retail nutrition records—capturing the behaviors of a large number of synthetically generated urban residents over a 12-week period. The framework integrates spatially explicit merchant selection using K-Dimensional (KD)-tree nearest-neighbor algorithms, temporally correlated anchor-based mobility simulation reflective of daily urban rhythms, and dietary-constraint filtering to preserve ecological validity in consumption patterns. In total, the system generates approximately 116 million mobility pings, 5.4 million transactions, and 1.9 million itemized purchases, yielding a reproducible benchmark for evaluating multimodal analytics, privacy-preserving computation, and secure IoT data-sharing protocols. To show the validity of this dataset, the underlying distributions of these residents were successfully validated against reported distributions in published research. We present preliminary uniqueness and cross-modal linkage indicators; comprehensive re-identification benchmarking against specific attack algorithms is planned as future work. This framework can be easily adapted to various scenarios of interest in Smart Cities and other IoT applications. By aligning methodological rigor with the operational needs of Smart City ecosystems, this work fills critical gaps in synthetic data generation for privacy-sensitive domains, including intelligent transportation systems, urban health informatics, and next-generation digital commerce infrastructures.

IoT↗

Identifying strengths and weaknesses of methods for computational network inference from single-cell RNA-seq data

Single-cell RNA-sequencing (scRNA-seq) offers unparalleled insight into the transcriptional programs of different cellular states by measuring the transcriptome of thousands of individual cells. An emerging problem in the analysis of scRNA-seq is the inference of transcriptional gene regulatory networks and a number of methods with different learning frameworks have been developed to address this problem. Here, we present an expanded benchmarking study of eleven recent network inference methods on seven published scRNA-seq datasets in human, mouse, and yeast considering different types of gold standard networks and evaluation metrics. We evaluate methods based on their computing requirements as well as on their ability to recover the network structure. We find that, while most methods have a modest recovery of experimentally derived interactions based on global metrics such as Area Under the Precision Recall curve, methods are able to capture targets of regulators that are relevant to the system under study. Among the top performing methods that use only expression were SCENIC, PIDC, MERLIN or Correlation. Addition of prior biological knowledge and the estimation of transcription factor activities resulted in the best overall performance with the Inferelator and MERLIN methods that use prior knowledge outperforming methods that use expression alone. We found that imputation for network inference did not improve network inference accuracy and could be detrimental. Comparisons of inferred networks for comparable bulk conditions showed that the networks inferred from scRNA-seq datasets are often better or at par with the networks inferred from bulk datasets. Our analysis should be beneficial in selecting methods for network inference. At the same time, this highlights the need for improved methods and better gold standards for regulatory network inference from scRNAseq datasets.

59 BASIC BIOLOGICAL SCIENCES↗

Multi-resolution Arctic Shrub Cover Dataset Derived from UAS and Airborne SfM and LiDAR (2013-2025)

We synthesized 177 unoccupied aerial system flights and 77 airborne flights across the Arctic and created a multi-resolution benchmark data of low-to-tall shrub fractional cover leveraging Structure-from-Motion and Light Detection and Ranging. The resulting dataset covered a total of 1899 km2 across Alaska, Western Canada, Sweden, and Siberian Arctic, including key sites from the Oro Arctic to the High Arctic. The dataset is organized into 6 primary data collection directories (“Abisko,” “AWI,” “ERE,” “Fairbanks,” “NGEE,” “Toolik”), each containing site and flight subdirectories. Flight directories include shrub cover rasters (*.tifs) at 1 m, 5 m, and 30 m resolution, the canopy height model at 1 m resolution (*.tifs), and a bounding box *.kml file. For the AWI, Abisko, NGEE, and Fairbanks collections, we also include the GCC raster at 1 m resolution (*.tif). Files are organized by Collection > Site > Flight Name > Data Files. Flight rasters are in the local UTM zone and the .kml files are in the geographic coordinate system EPSG 4326. We also include a .csv file that details the source datasets for every flight. The Next-Generation Ecosystem Experiments in the Arctic (NGEE Arctic) project is a research effort to reduce uncertainty in the Department of Energy’s Energy Exascale Earth System Model (E3SM) by developing a predictive understanding of Arctic tundra ecosystems underlain by permafrost and to quantify feedbacks from the Arctic tundra to the Earth system. NGEE Arctic is supported by the Department of Energy's Office of Biological and Environmental Research. Over Phases 1–3, observations made by the NGEE Arctic team across a gradient of permafrost landscapes in Arctic Alaska improved the representation of tundra processes in the land surface component of E3SM (the E3SM Land Model, ELM). Model improvements emphasized unique aspects of permafrost environments and explored reductions in model complexity while retaining predictive power. The Arctic-informed ELM developed by NGEE Arctic has been used to make novel predictions on processes ranging from permafrost thaw to soil biogeochemical cycling to Earth system feedbacks associated with the unique characteristics of tundra plants. In Phase 4, the NGEE Arctic team is evaluating our new predictive understanding under novel conditions across the Arctic domain. In collaboration with partners at long-term pan-Arctic research sites we are examining whether an Arctic-informed ELM can faithfully simulate interactions among surface and subsurface processes at site, regional, and pan-Arctic scales. In turn, we are using variety of tools to dynamically extend and evaluate ELM inference, with an emphasis on data synthesis and pan-Arctic model evaluation, reintegration of code with an evolving E3SM, scaling across heterogeneous Arctic landscapes, and the appropriate representation of the impacts of increasingly frequent Arctic disturbances.

canopy height model↗

AutoTandemML: Active Learning Enhanced Tandem Neural Networks for Inverse Design Problems

Inverse design in science and engineering involves determining optimal design parameters that achieve desired performance outcomes, a process often hindered by the complexity and high dimensionality of design spaces, leading to significant computational costs. To tackle this challenge, we propose a novel hybrid approach that combines active learning with Tandem Neural Networks to enhance the efficiency and effectiveness of solving inverse design problems. Active learning allows to selectively sample the most informative data points, reducing the required dataset size without compromising accuracy. We investigate this approach using three benchmark problems: airfoil inverse design, photonic surface inverse design, and scalar boundary condition reconstruction in diffusion partial differential equations. We demonstrate that integrating active learning with Tandem Neural Networks outperforms standard approaches across the benchmark suite, achieving better accuracy with fewer training samples.

97 MATHEMATICS AND COMPUTING↗

Discovering type I cis-AT polyketides through computational mass spectrometry and genome mining with Seq2PKS

Type 1 polyketides are a major class of natural products used as antiviral, antibiotic, antifungal, antiparasitic, immunosuppressive, and antitumor drugs. Analysis of public microbial genomes leads to the discovery of over sixty thousand type 1 polyketide gene clusters. However, the molecular products of only about a hundred of these clusters are characterized, leaving most metabolites unknown. Characterizing polyketides relies on bioactivity-guided purification, which is expensive and time-consuming. To address this, we present Seq2PKS, a machine learning algorithm that predicts chemical structures derived from Type 1 polyketide synthases. Seq2PKS predicts numerous putative structures for each gene cluster to enhance accuracy. The correct structure is identified using a variable mass spectral database search. Benchmarks show that Seq2PKS outperforms existing methods. Applying Seq2PKS to Actinobacteria datasets, we discover biosynthetic gene clusters for monazomycin, oasomycin A, and 2-aminobenzamide-actiphenol.

60 APPLIED LIFE SCIENCES↗

Automatic Crack Segmentation and Feature Extraction in Electroluminescence Images of Solar Modules

The effect of cracks in solar cells on the long-term degradation of photovoltaic (PV) modules remains to be determined. To investigate this effect in future studies, it is necessary to quantitatively describe the crack features (e.g., length) and correlate them with module power loss. Electroluminescence (EL) imaging is a common technique for identifying cracks. However, it is currently challenging and time-consuming to identify cracks in a large number of EL images and quantify complex crack features by human inspection. This article introduces a fast semantic segmentation method (~0.18 s/cell) to automatically segment cracks from EL images and algorithms to extract crack features. Here we fine-tuned a UNet neural network model using pretrained VGG16 as the encoder and obtained an average F1 score of 0.875 and an intersection over union score of 0.782 on the testing set. With cracks and busbars segmented, we developed algorithms for extracting crack features, including the crack-isolated area, the brightness inside the isolated area, and the crack length. We also developed an automatic preprocessing tool for cropping individual cell images from EL images of PV modules (~0.72 s/module). Our codes are published as open-source an software, and our annotated dataset composed of various types of cells is published as a benchmark for crack segmentation in EL images.

14 SOLAR ENERGY↗

Emerging Flexible Designs for Geospatial Multimodal Foundation Models

Foundation models are rapidly transforming Earth observation by enabling scalable pretraining across diverse unlabeled geospatial modalities. However, their architectural diversity—ranging from encoder-only to encoder-decoder and masked autoencoding paradigms—makes it challenging to assess performance trade-offs in a consistent manner. In this work, we present an apples-to-apples comparison of leading FM architectures designed for geospatial multimodal reasoning, with a particular focus on flexibility across varied spectral band configurations. We standardize pretraining using identical self-supervised learning objectives and training datasets, and evaluate all models under consistent parameterization on the GEOBench benchmark across classification and segmentation tasks. Our results offer new insights into the design trade-offs between model flexibility, modality alignment, and downstream task performance. By highlighting architectural strengths and limitations under controlled conditions, this study provides practical guidance for building next-generation geospatial foundation models capable of robust multimodal reasoning.

Ambrozio Dias, Philipe [ORNL] (ORCID:0000000194277↗

Simple and efficient algorithms for training machine learning potentials to force data

Machine learning models, trained on data from ab initio quantum simulations, are yielding molecular dynamics potentials with unprecedented accuracy. One limiting factor is the quantity of available training data, which can be expensive to obtain. A quantum simulation often provides all atomic forces, in addition to the total energy of the system. These forces provide much more information than the energy alone. It may appear that training a model to this large quantity of force data would introduce significant computational costs. Actually, training to all available force data should only be a few times more expensive than training to energies alone. Here, we present a new algorithm for efficient force training, and benchmark its accuracy by training to forces from real-world datasets for organic chemistry and bulk aluminum.

74 ATOMIC AND MOLECULAR PHYSICS↗

CHQ- SocioEmo: Identifying Social and Emotional Support Needs in Consumer-Health Questions

General public, often called consumers, are increasingly seeking health information online. To be satisfactory, answers to health-related questions often have to go beyond informational needs. Automated approaches to consumer health question answering should be able to recognize the need for social and emotional support. Recently, large scale datasets have addressed the issue of medical question answering and highlighted the challenges associated with question classification from the standpoint of informational needs. However, there is a lack of annotated datasets for the non-informational needs. We introduce a new dataset for non-informational support needs, called CHQ-SocioEmo. The Dataset of Consumer Health Questions was collected from a community question answering forum and annotated with basic emotions and social support needs. This is the first publicly available resource for understanding non-informational support needs in consumer health-related questions online. We benchmark the corpus against multiple state-of-the-art classification models to demonstrate the dataset’s effectiveness.

60 APPLIED LIFE SCIENCES↗

Systematic Evaluation of Atmospheric Forcing, Surface Datasets, and Mesh Effects on Kilometer-Scale Land Surface and River Modeling

Earth system models are advancing toward kilometer-scale resolution to capture local climate impacts and extremes. High-resolution land and river modeling depends on multiple factors, including mesh, surface datasets, and atmospheric forcing, but their relative effects at kilometer scales remain unquantified. We evaluated five Energy Exascale Earth System Model land and river configurations over the Mid-Atlantic region using two mesh (1/8° structured versus variable-resolution unstructured mesh), two surface datasets (default versus newly developed), and three atmospheric forcings (NLDAS2, MSWX, GSWP). Evaluation against satellite, reanalysis, and in situ benchmarks across water, energy, and carbon cycles quantifies how these factors affect model performance. Forcing selection produces the largest bias reductions (12-99% across variables), followed by surface datasets (7-75%) and mesh (up to 21%). Forcing effects vary by variable, with MSWX reducing biases for snow water equivalent, evapotranspiration, albedo, temperature, and gross primary productivity, GSWP for snow cover and runoff, and NLDAS for soil moisture and streamflow. The use of newly developed surface datasets improves gross primary productivity (58% bias reduction) and evapotranspiration but increase soil moisture and albedo biases due to current modeling limitations. Variable-resolution unstructured mesh improves the simulation of small-basin streamflow through better capturing drainage networks, though mesh minimally affects other land variables. These findings provide important guidance for high-resolution modeling development and actionable science.

Land and River modeling↗

On-Demand Column Joining for High Energy Physics

As the Large Hadron Collider (LHC) transitions into the High-Luminosity LHC (HL-LHC) era, the volume of data to be processed is expected to increase significantly. The CMS Experiment currently utilizes various data formats, including AOD, MiniAOD, and NanoAOD, each with different levels of detail and storage requirements. This paper addresses the challenges of data duplication and storage inefficiencies in high-energy physics (HEP) analyses by proposing an on-demand column-joining solution. This approach aims to reduce data duplication by enabling the dynamic combination of NanoAOD data with auxiliary information from larger data tiers, such as MiniAOD. The proposed solution leverages Trino, a high-performance distributed SQL query engine, to perform efficient and scalable data joins. Benchmarks using CMS OpenData demonstrate the feasibility of this approach, showing that it can handle large datasets with low latency. Integration with the scikit-hep ecosystem and the coffea analysis framework is also discussed, highlighting the potential for seamless end-to-end data processing and analysis. Ongoing and future work focuses on expanding benchmarks, integrating ServiceX for data transformation, and exploring the use of native object storage solutions.

Manganelli, Nicholas [Northeastern U.]↗

An Autonomous MCP Bridge to Rucio: Enhancing Data Management Accessibility for High Energy Physics

The Rucio Data Management System [1] is an important tool used by High Energy Physics experiments, including those at Fermi National Accelerator Laboratory, to store and manage exabyte-scale scientific datasets. Despite its central role in coordinating data across globally distributed storage sites, Rucio's command line interface (CLI) presents a steep learning curve, and makes it difficult for scientists to navigate through. To solve this issue, a containerized Model Context Protocol (MCP) [2] server was built that connects Large Language Models directly to Rucio, allowing AI agents to handle data tasks by using simple, natural language rather than memorized terminal commands. The core engineering focus of this project was moving the server away from slow terminal commands that require text parsing and replacing them with a native Python Client API toolset and a planned REST API framework. Moving to the Python API handles data operations directly in memory, which helps clear up formatting errors, provides the AI with clean, structured JSON data and speeds up tool execution. To prove that the system actually works, a benchmarking pipeline was also built with various questions to test the AI across four different model configurations. The questions included finding data scopes, tracking down specific datasets, and checking replication rules. Through benchmarking, early runs showed that with raw terminal text, the model would get confused and stuck, whereas switching to the Python API to feed the AI clean, structured data yielded massive improvement. By creating an intelligent and autonomous bridge to a storage network, this project shows how AI can be implemented in scientific data management, which ultimately helps scientists at Fermilab spend less time sorting through data and more time focusing on their experiments and analysis.

Akella, Kashyap [William Rainey Harper Coll.]↗

The Superfund Research Program Analytics Portal: linking environmental chemical exposure to biological phenotypes

The OSU/PNNL Superfund Research Program represents a longstanding collaboration to quantify Polycyclic Aromatic Hydrocarbons (PAHs) at various superfund sites in the Pacific Northwest and assess their potential impact on human health. To link the chemical measurements to biological activity, we describe the use of the zebrafish as a high-throughput developmental model of human exposure that provides quantitative measurements of the biological ramifications of exposure to toxicants. Toward this end, we have linked over 150 PAHs found at Superfund sites to the effect of these same chemicals in zebrafish, creating a rich dataset that links environmental exposure to biological response. To quantify this response, we have implemented a dose-response modelling pipeline to calculate benchmark dose parameters which enable potency comparison across over 500 chemicals and 10 zebrafish-specific phenotypes. Our portal provides public access to this dataset via an interactive web site designed to support exploration and re-use of these data by the scientific community at http://srp.pnnl.gov.

Gosline, Sara JC↗

Maps of growing season gross primary production and net ecosystem exchange for Council Road Mile Marker 71, Seward Peninsula, Alaska, [2017-2023]

This data archive is in support of the Next-Generation Ecosystem Experiments in the Arctic (NGEE Arctic) publication "Integrating Characteristic Arctic Vegetation in a Land Surface Model Improves Representation of Carbon Dynamics Across a Tundra Landscape", by Murphy et al. (2025a). Murphy et al. (2025a) evaluated whether incorporating observed Arctic vegetation heterogeneity into ELM, the land model of the Department of Energy’s Energy Exascale Earth System Model (E3SM), improved simulations of tundra carbon cycling. The associated model archive can be found at Murphy et al. (2025b). The study focused on the spatial patterns and net landscape-level growing season productivity and carbon uptake. As part of this evaluation, observationally derived maps of average growing season (June–August) net ecosystem exchange (NEE) and gross primary production (GPP) were developed for the same domain. These maps, which form the dataset described here, integrate eddy covariance flux tower, remote sensing, and vegetation community data to provide spatially explicit benchmarks for model evaluation. The maps provide spatially explicit estimates of average growing season NEE and GPP across 13 tundra vegetation communities within the study domain. By combining flux tower observations with Airborne Visible-Infrared Imaging Spectrometer-Next Generation (AVIRIS-NG) hyperspectral imagery and drone-based normalized difference vegetation index (NDVI), these maps capture the heterogeneity of carbon fluxes associated with different Arctic vegetation types. While they represent average seasonal conditions rather than interannual variability, the maps provide a unique dataset for evaluating model performance, comparing vegetation community contributions to landscape-scale carbon cycling, and supporting regional analyses of Arctic carbon dynamics. This data archive contains 5 m resolution maps of vegetation communities, vegetation community average growing season GPP, and vegetation community average growing season NEE (three *.tif files), a User’s Guide (*pdf file), and Table 1 of the User’s Guide displaying vegetation community coverage and average growing season NEE and GPP values (*.csv file).

Murphy, Bailey [ORNL] (ORCID:0000000203995221)↗

Benchmarking Soft Sensors for Remote Monitoring of On-Site Wastewater Treatment Plants

On-site wastewater treatment plants (OSTs) are usually unattended, so failures often remain undetected and lead to prolonged periods of reduced performance. To stabilize the performance of unattended plants, soft sensors could expose faults and failures to the operator. In a previous study, we developed soft sensors and showed that soft sensors with data from unmaintained physical sensors can be as accurate as soft sensors with data from maintained ones. The monitored variables were pH and dissolved oxygen (DO), and soft sensors were used to predict nitrification performance. In the present study, we use synthetic data and monitor three plants to test these soft sensors. We find that a long solids retention time and a moderate aeration rate improve the pH soft-sensor accuracy and that the aeration regime is the main operational parameter affecting the accuracy of the DO soft sensor. We demonstrate that integrated design of monitoring and control is necessary to achieve robustness when extrapolating from one OST to another in the absence of plant-specific fine-tuning. Additionally, we provide a unique labeled dataset for further feature and data-driven soft-sensor development. Our benchmarking results indicate that it is feasible to monitor OSTs with unmaintained sensors and without plant-specific tuning of the developed soft sensors. This is expected to drastically reduce monitoring costs for OST-based sanitation systems.

54 ENVIRONMENTAL SCIENCES↗

Land Model Testbed: Accelerating Development, Benchmarking and Analysis of Land Surface Models

A Land Model Testbed (LMT), designed to provide a computational framework for systematically assessing model fidelity and supporting rapid development of complex multiscale models, offers a general-purpose workflow for conducting large ensemble simulations of multiple land surface models, post-processing large volumes of model output, and evaluating model results. It leverages existing tools for launching model simulations and the International Land Model Benchmarking (ILAMB) package for assessing model fidelity through comparison with best-available observational datasets. Increased complexity and proliferation of uncertain parameters in process representations in land surface models has driven the need for frequent and intensive testing and evaluating of models to quantify uncertainties and optimize parameters such that results are consistent with observations. The LMT described here meets these needs by providing tools to run thousands of ensemble simulations simultaneously and post-process their output files, by automating execution of an enhanced version of ILAMB with site-specific benchmarks and multivariate functional relationships, and by offering ensemble diagnostics and a customizable dashboard for displaying model performance metrics and associated graphics. We envision the LMT capabilities will serve as a foundational computational resource for a proposed user facility focused on terrestrial multiscale model--data integration.

Sreepathi, Sarat↗