Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS2023

A Blind Convolutional Deep Autoencoder for Spectral Unmixing of Hyperspectral Images Over Waterbodies

Harmful algal blooms have dangerous repercussions for biodiversity, the ecosystem, and public health. Automatic identification based on remote sensing hyperspectral image analysis provides a valuable mechanism for extracting the spectral signatures of harmful algal blooms and their respective percentage in a region of interest. This paper proposes a new model called a non-symmetrical autoencoder for spectral unmixing to perform endmember extraction and fractional abundance estimation. The model is assessed in benchmark datasets, such as Jasper Ridge and Samson. Additionally, a case study of the HSI2 image acquired by NASA over Lake Erie in 2017 is conducted for extracting optical water types. The results using the proposed model for the benchmark datasets improve unmixing performance, as indicated by the spectral angle distance compared to five baseline algorithms. Improved results were obtained for various metrics. In the Samson dataset, the proposed model outperformed other methods for water (0.060) and soil (0.025) endmember extraction. Moreover, the proposed method exhibited superior performance in terms of mean spectral angle distance compared to the other five baseline algorithms. The non-symmetrical autoencoder for the spectral unmixing approach achieved better results for abundance map estimation, with a root mean square error of 0.091 for water and 0.187 for soil, compared to the ground truth. For the Jasper Ridge dataset, the non-symmetrical autoencoder for the spectral unmixing model excelled in the tree (0.039) and road (0.068) endmember extraction and also demonstrated improved results for water abundance maps (0.1121). The proposed model can identify the presence of chlorophyll-a in waterbodies. Chlorophyll-a is an essential indicator of the presence of the different concentrations of macrophytes and cyanobacteria. The non-symmetrical autoencoder for spectral unmixing achieves a value of 0.307 for the spectral angle distance metric compared to a reference ground truth spectral signature of chlorophyll-a. The source code for the proposed model, as implemented in this manuscript, can be found at https://github.com/EstefaniaAlfaro/autoencoder_owt_spectral.git.

hyperspectral imaging↗
DOE OSTI2022

The Dark Machines Anomaly Score Challenge: Benchmark Data and Model Independent Event Classification for the Large Hadron Collider

We describe the outcome of a data challenge conducted as part of the Dark Machines (https://www.darkmachines.org) initiative and the Les Houches 2019 workshop on Physics at TeV colliders. The challenged aims to detect signals of new physics at the Large Hadron Collider (LHC) using unsupervised machine learning algorithms. First, we propose how an anomaly score could be implemented to define model-independent signal regions in LHC searches. We define and describe a large benchmark dataset, consisting of >1 billion simulated LHC events corresponding to 10\, fb^{-1} 10 f b − 1 of proton-proton collisions at a center-of-mass energy of 13 TeV. We then review a wide range of anomaly detection and density estimation algorithms, developed in the context of the data challenge, and we measure their performance in a set of realistic analysis environments. We draw a number of useful conclusions that will aid the development of unsupervised new physics searches during the third run of the LHC, and provide our benchmark dataset for future studies at https://www.phenoMLdata.org. Code to reproduce the analysis is provided at https://github.com/bostdiek/DarkMachines-UnsupervisedChallenge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Evaluation and improvement of the parameterization of aerosol hygroscopicity in global climate models using in-situ surface measurements (Final Report)

Aerosols are tiny particles suspended on the atmosphere that can interact with incoming solar radiation and affect the Earth radiative budget. They do so by scattering and absorbing solar radiation, and these properties vary depending on the aerosol size and chemical composition. Moreover, by taking up water from the surrounding air, hygroscopic aerosol particles will grow in size and change their chemical composition, thus modifying their optical properties (scattering and absorption) and their final impact on radiative forcing calculations. An accurate knowledge of aerosol hygroscopicity is crucial for estimating the net radiative impact of aerosols. We took a three-pronged approach to improve our understanding of aerosol hygroscopicity and how it is implemented in Earth system models. In the first part of our project, we developed a benchmark dataset from existing aerosol hygroscopic growth measurements made by tandem nephelometer humidogram systems. We analyzed, using a standardized methodology, observations from 26 in-situ stations around the globe. Measurement data was collected from multiple data providers, reviewed and harmonized to create a consistent dataset of the scattering enhancement factor due to aerosol water uptake. This dataset is archived in several publicly available databases for use by interested researchers. In the second part of the project, we used the benchmark hygroscopicity dataset to perform a global study on aerosol hygroscopicity and aerosol optical properties. Measurements show a global picture of scattering enhancement with larger values for Arctic and marine sites and lower for urban and desert sites. We assessed the RH dependence of aerosol radiative forcing and showed that the overall effect of aerosol hygroscopicity on DARF is an increase in the absolute forcing effect (negative sign) by a factor of up to 4 compared to dry conditions (RH<40%). Finally, we explored using aerosol single scattering albedo (SSA) and scattering Angstrom exponent (SAE) as possible proxies for aerosol hygroscopicity. SSA showed more promise as a surrogate for the scattering enhancement factor than SAE, but neither was ideal. In the third part of the project we evaluated the output of ten Earth system models (ESMs) against the benchmark hygroscopicity dataset. ESMs utilize various schemes for aerosol hygroscopicity which had not been previously tested against observations on a global scale. We found that ESMs currently overestimate scattering enhancement due to hygroscopic growth. Model parameterizations of hygroscopicity and model chemistry are two main factors driving the observed diversity in hygroscopicity simulations among the models. In addition, our study makes several suggestions for modelers, including improving the parameterizations of organic and sea salt aerosol hygroscopicity. Future hygroscopicity model evaluation experiments should include the model data related to particles size which was not available for our study.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

A Molecular-MNIST Dataset for Machine Learning Study on Diffraction Imaging and Microscopy

An image dataset of 10 different size molecules, where each molecule has 2,000 structural variants, is generated from the 2D cross-sectional projection of Molecular Dynamics trajectories. The purpose of this dataset is to provide a benchmark dataset for the increasing need of machine learning, deep learning and image processing on the study of scattering, imaging and microscopy.

BASIC BIOLOGICAL SCIENCES,OTHER INSTRUMENTATION↗
DOE OSTI2023

MArVD2: a machine learning enhanced tool to discriminate between archaeal and bacterial viruses in viral datasets

Abstract Our knowledge of viral sequence space has exploded with advancing sequencing technologies and large-scale sampling and analytical efforts. Though archaea are important and abundant prokaryotes in many systems, our knowledge of archaeal viruses outside of extreme environments is limited. This largely stems from the lack of a robust, high-throughput, and systematic way to distinguish between bacterial and archaeal viruses in datasets of curated viruses. Here we upgrade our prior text-based tool (MArVD) via training and testing a random forest machine learning algorithm against a newly curated dataset of archaeal viruses. After optimization, MArVD2 presented a significant improvement over its predecessor in terms of scalability, usability, and flexibility, and will allow user-defined custom training datasets as archaeal virus discovery progresses. Benchmarking showed that a model trained with viral sequences from the hypersaline, marine, and hot spring environments correctly classified 85% of the archaeal viruses with a false detection rate below 2% using a random forest prediction threshold of 80% in a separate benchmarking dataset from the same habitats.

Vik, Dean (ORCID:000000027546899X)↗
DOE OSTI2023

Characterizing Quantum Classifier Utility in Natural Language Processing Workflows

Quantum Natural Language Processing (QNLP) develops natural language processing (NLP) models for deployment on quantum computers. We explore feature and data prototype selection techniques to address challenges posed by encoding high dimensional features. Our study builds quantum circuit classifiers that includes classical feature pre-processing, quantum embedding and quantum model training. The quantum models are built on 4 or 6 qubits and the quantum neural network (QNN) uses the established bricklayer design. We compare the dependence of model performance (in terms of accuracy and F1 scores) on feature length, embedding gates and parameterized unitary design. We compare the performance of quantum machine learning models to classical convolution neural network model (CNN) on binary and multi-class classification tasks using two datasets of synthetic features and labels. The first is the ECP-CANDLE P3B3 dataset a corpus of synthetically generated cancer pathology reports. The second dataset is extracted from well-known benchmark dataset (MADELON) - features are generated with a combination of informative, repeated and uninformative features. Both datasets are used for binary classification and multi-class classification with 3 classes. We observe robust, accurate performance from all models on the binary classification tasks, but multiclass classification is a challenge for the quantum models-there is a notable decrease in accuracy when using 3 classes. Overall the performance is comparable in terms of recall and accuracy between QNNs and CNNs, even with large datasets. These results provide a point of comparison between quantum and classical models on real-world datasets.

Hamilton, Kathleen↗
DOE OSTI2021

The LAKE model input dataset for three Arctic lakes

This dataset contains meteorological data collected for three Arctic lakes and compiled to satisfy input requirements of the LAKE 2.0 model. The dataset was generated to act as a benchmarking dataset for future model-data inter-comparisons. The LAKE 2.0 model simulates temperatures within the water later and the sedimentary layer of a lake. The LAKE2.0. is an open-source code and available to download via this weblike http://tesla.parallel.ru/Viktor/LAKE/-/wikis/LAKE-model (last visit July 14, 2021). The meteorological data are required to simulate the surface energy balance at the surface of a lake. This dataset includes a compilation of the meteorological data pulled from multiple data streams, including National Oceanic and Atmospheric Administration (NOAA) climate data, Circumarctic Lakes Observation Network (CALON) data, and the United States Geological Survey (USGS) data. The data were compiled for three Arctic lakes: FoxDen (66.55877, -164.45670), Atqasuk (70.452497, -156.951984), and Toolik (68.63150, -149.60740). Each meteorological data is in comma-delimited format (file extension ‘.dat’) and includes eight columns: Temperature [K], Pressure [Pa], longwave downward radiation [W/m2], shortwave downward radiation [W/m2], “U” wind speed [m/s], ”V” wind speed [m/s], humidity [kg/kg], precipitation [m/s]. In addition to the meteorological data file, we included setup and driver files. The Toolik lake is the deepest out of three lakes and has inflowing and outflowing groundwater data. InflowOutflowREADME.txt has more information about inflow and outflow flies. The other two lakes are much shallower and modeled as a closed system (i.e. no water inflow or outflow).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

A consensus-based ensemble approach to improve transcriptome assembly

Systems-level analyses, such as differential gene expression analysis, co-expression analysis, and metabolic pathway reconstruction, depend on the accuracy of the transcriptome. Multiple tools exist to perform transcriptome assembly from RNAseq data. However, assembling high quality transcriptomes is still not a trivial problem. This is especially the case for non-model organisms where adequate reference genomes are often not available. Different methods produce different transcriptome models and there is no easy way to determine which are more accurate. Furthermore, having alternative-splicing events exacerbates such difficult assembly problems. While benchmarking transcriptome assemblies is critical, this is also not trivial due to the general lack of true reference transcriptomes. In this study, we first provide a pipeline to generate a set of the simulated benchmark transcriptome and corresponding RNAseq data. Using the simulated benchmarking datasets, we compared the performance of various transcriptome assembly approaches including both de novo and genome-guided methods. The results showed that the assembly performance deteriorates significantly when alternative transcripts (isoforms) exist or for genome-guided methods when the reference is not available from the same genome. To improve the transcriptome assembly performance, leveraging the overlapping predictions between different assemblies, we present a new consensus-based ensemble transcriptome assembly approach, ConSemble. Without using a reference genome, ConSemble using four de novo assemblers achieved an accuracy up to twice as high as any de novo assemblers we compared. When a reference genome is available, ConSemble using four genome-guided assemblies removed many incorrectly assembled contigs with minimal impact on correctly assembled contigs, achieving higher precision and accuracy than individual genome-guided methods. Furthermore, ConSemble using de novo assemblers matched or exceeded the best performing genome-guided assemblers even when the transcriptomes included isoforms. We thus demonstrated that the ConSemble consensus strategy both for de novo and genome-guided assemblers can improve transcriptome assembly. The RNAseq simulation pipeline, the benchmark transcriptome datasets, and the script to perform the ConSemble assembly are all freely available from: http://bioinfolab.unl.edu/emlab/consemble/.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Fourier-DeepONet: Fourier-enhanced deep operator networks for full waveform inversion with improved accuracy, generalizability, and robustness

In this article, full waveform inversion (FWI) infers the subsurface structure information from seismic waveform data by solving a non-convex optimization problem. Data-driven FWI has been increasingly studied with various neural network architectures to improve accuracy and computational efficiency. Nevertheless, the applicability of pre-trained neural networks is severely restricted by potential discrepancies between the source function used in the field survey and the one utilized during training. Here, we develop a Fourier-enhanced deep operator network (Fourier-DeepONet) for FWI with the generalization of seismic sources, including the frequencies and locations of sources. Specifically, we employ the Fourier neural operator as the decoder of DeepONet, and we utilize source parameters as one input of Fourier-DeepONet, facilitating the resolution of FWI with variable sources. To test Fourier-DeepONet, we develop three new and realistic FWI benchmark datasets (FWI-F, FWI-L, and FWI-FL) with varying source frequencies, locations, or both. Our experiments demonstrate that compared with existing data-driven FWI methods, Fourier-DeepONet obtains more accurate predictions of subsurface structures in a wide range of source parameters. Moreover, the proposed Fourier-DeepONet exhibits superior robustness when handling data with Gaussian noise or missing traces and sources with Gaussian noise, paving the way for more reliable and accurate subsurface imaging across diverse real conditions.

42 ENGINEERING↗
DOE OSTI2022

CLM5 CAMELS Basins Ensemble

Land surface models such as Community Land Model Version 5 (CLM5) are essential tools for simulating the behaviors of the terrestrial system. Despite the extensive application of CLM5, limited attention has been paid to the underlying uncertainties associated with its hydrologic parameters and the implications that these uncertainties have on water resources applications. To address this long-standing issue, we conduct a comprehensive hydrologic parameter uncertainty characterization (UC) of CLM5 over the hydroclimatic gradients of the Conterminous United States using five meteorological datasets. Key datasets produced from the UC experiment include a benchmark dataset of CLM5 default hydrological performance, parameter sensitivity identified for 28 hydrological metrics, and large ensemble outputs for hydrological predictions. The presented datasets can assist CLM5 calibration and to support broad applications such as evaluating vulnerabilities to droughts and floods. The dataset can be used to identify under what hydroclimate conditions parametric uncertainties demonstrate substantial effects on hydrological predictions and clarify where further investigations are needed to understand how land runoff uncertainties interact with other Earth system processes. Please refer to the included README.pdf for a description of the included files. Note that raw CLM5 model outputs for each forcing dataset are hosted on a Globus endpoint: https://app.globus.org/file-manager?destination_id=d22ef858-27b0-11ed-a910-fd3165076336.

Yan, Hongxiang↗
DOE OSTI2023

CLM5 CAMELS Basins Ensemble

Land surface models such as Community Land Model Version 5 (CLM5) are essential tools for simulating the behaviors of the terrestrial system. Despite the extensive application of CLM5, limited attention has been paid to the underlying uncertainties associated with its hydrologic parameters and the implications that these uncertainties have on water resources applications. To address this long-standing issue, we conduct a comprehensive hydrologic parameter uncertainty characterization (UC) of CLM5 over the hydroclimatic gradients of the Conterminous United States using five meteorological datasets. Key datasets produced from the UC experiment include a benchmark dataset of CLM5 default hydrological performance, parameter sensitivity identified for 28 hydrological metrics, and large ensemble outputs for hydrological predictions. The presented datasets can assist CLM5 calibration and to support broad applications such as evaluating vulnerabilities to droughts and floods. The dataset can be used to identify under what hydroclimate conditions parametric uncertainties demonstrate substantial effects on hydrological predictions and clarify where further investigations are needed to understand how land runoff uncertainties interact with other Earth system processes. Please refer to the README.pdf for a description of the included files. Note that raw CLM5 model outputs for each forcing dataset are hosted on a Globus endpoint: https://app.globus.org/file-manager?destination_id=d22ef858-27b0-11ed-a910-fd3165076336.

Yan, Hongxiang↗
DOE OSTI2023

Characterizing uncertainty in Community Land Model version 5 hydrological applications in the United States

Abstract Land surface models such as the Community Land Model Version 5 (CLM5) are essential tools for simulating the behavior of the terrestrial system. Despite the extensive application of CLM5, limited attention has been paid to the underlying uncertainties associated with its hydrological parameters and how these uncertainties affect water resource applications. To address this long-standing issue, we use five meteorological datasets to conduct a comprehensive hydrological parameter uncertainty characterization of CLM5 over the hydroclimatic gradients of the conterminous United States. Key datasets produced from the uncertainty characterization experiment include: a benchmark dataset of CLM5 default hydrological performance, parameter sensitivities for 28 hydrological metrics, and large-ensemble outputs for CLM5 hydrological predictions. The presented datasets will assist CLM5 calibration and support broad applications, such as evaluating drought and flood vulnerabilities. The datasets can be used to identify the hydroclimatological conditions under which parametric uncertainties demonstrate substantial effects on hydrological predictions and clarify where further investigations are needed to understand how hydrological prediction uncertainties interact with other Earth system processes.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Spin-informed universal graph neural networks for simulating magnetic ordering

The screening and discovery of magnetic materials are hindered by the computational cost of first-principles density-functional theory (DFT) calculations required to find the ground state magnetic ordering. Although universal machine-learning interatomic potentials (uMLIPs), also known as atomistic foundation models, offer high-fidelity models of many atomistic systems with significant speedup, they currently lack the inputs required for predicting magnetic ordering. In this work, we present a data-efficient, spin-informed graph neural network framework that incorporates spin degrees of freedom as inputs and preserves physical symmetries, extending the functionality of uMLIPs to simulate magnetic orderings. This framework speeds up DFT calculations through better initial guesses for magnetic moments, determines the ground-state ordering of bulk materials and even generalizes to magnetic ordering in surfaces. Furthermore, we implement a closed-loop anomaly detection approach that effectively addresses the classic "chicken-and-egg" problem of creating a high-quality dataset while developing a uMLIP, unearthing anomalies in large benchmark datasets and boosting model accuracy.

Xu, Wenbin↗
DOE OSTI2023

Review of machine learning and deep learning models for toxicity prediction

The ever-increasing number of chemicals has raised public concerns due to their adverse effects on human health and the environment. To protect public health and the environment, it is critical to assess the toxicity of these chemicals. Traditional in vitro and in vivo toxicity assays are complicated, costly, and time-consuming and may face ethical issues. These constraints raise the need for alternative methods for assessing the toxicity of chemicals. Recently, due to the advancement of machine learning algorithms and the increase in computational power, many toxicity prediction models have been developed using various machine learning and deep learning algorithms such as support vector machine, random forest, k-nearest neighbors, ensemble learning, and deep neural network. This review summarizes the machine learning- and deep learning-based toxicity prediction models developed in recent years. Support vector machine and random forest are the most popular machine learning algorithms, and hepatotoxicity, cardiotoxicity, and carcinogenicity are the frequently modeled toxicity endpoints in predictive toxicology. It is known that datasets impact model performance. The quality of datasets used in the development of toxicity prediction models using machine learning and deep learning is vital to the performance of the developed models. The different toxicity assignments for the same chemicals among different datasets of the same type of toxicity have been observed, indicating benchmarking datasets is needed for developing reliable toxicity prediction models using machine learning and deep learning algorithms. This review provides insights into current machine learning models in predictive toxicology, which are expected to promote the development and application of toxicity prediction models in the future.

Research & Experimental Medicine↗
DOE OSTI2025

Mic-hackathon 2024: hackathon on machine learning for electron and scanning probe microscopy

Microscopy is one of the primary sources of information on materials structure and functionality at the nanometer and atomic scales. The data generated through microscopy is often contained in well-structured datasets, enriched with extensive metadata and sample histories, although not always with the same level of detail or storage format. The broad incorporation of data management plans by major funding agencies ensures the preservation and accessibility of this data. However, deriving insights from these rich datasets remains challenging due to the lack of established code ecosystems, standardized benchmarks, and integration strategies. Correspondingly, the efficiency of data usage is very low, and time expenditures at the analysis stage are enormous. In addition to post-acquisition data analysis, the emergence of application programming interfaces by major microscope manufacturers now creates opportunities for real-time ML-based data analytics to enable automated decision making, and particularly ML-agent controlled real-time microscope operation. Despite these opportunities, there is a significant gap in integrating the ML community with the broader microscopy community, limiting the value that these methods bring to physics and materials discovery and materials optimization. Hackathons address these challenges by fostering collaboration between ML experts and microscopy professionals, encouraging the development of innovative solutions that leverage ML for microscopy and preparing the workforce of the future both for microscopy-intensive domains areas, instrument manufacturers, and ML scientists interested in real world applications for fundamental research, materials optimization, and manufacturing. The hackathon generated benchmark datasets and digital twins of microscopes that further contribute to the development of the field and establish data analysis ecosystems. All the codes can be found at GitHub(https://github.com/KalininGroup/Mic-hackathon-2024-codes-publication/tree/1.0.0.1) and Zenodo (https://zenodo.org/records/15579940).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Forward variable selection enables fast and accurate dynamic system identification with Karhunen-Loève decomposed Gaussian processes

A promising approach for scalable Gaussian processes (GPs) is the Karhunen-Loève (KL) decomposition, in which the GP kernel is represented by a set of basis functions which are the eigenfunctions of the kernel operator. Such decomposed kernels have the potential to be very fast, and do not depend on the selection of a reduced set of inducing points. However KL decompositions lead to high dimensionality, and variable selection thus becomes paramount. This paper reports a new method of forward variable selection, enabled by the ordered nature of the basis functions in the KL expansion of the Bayesian Smoothing Spline ANOVA kernel (BSS-ANOVA), coupled with fast Gibbs sampling in a fully Bayesian approach. It quickly and effectively limits the number of terms, yielding a method with competitive accuracies, training and inference times for tabular datasets of low feature set dimensionality. Theoretical computational complexities are O ( N P 2 ) in training and O ( P ) per point in inference, where N is the number of instances and P the number of expansion terms. The inference speed and accuracy makes the method especially useful for dynamic systems identification, by modeling the dynamics in the tangent space as a static problem, then integrating the learned dynamics using a high-order scheme. The methods are demonstrated on two dynamic datasets: a ‘Susceptible, Infected, Recovered’ (SIR) toy problem, along with the experimental ‘Cascaded Tanks’ benchmark dataset. Comparisons on the static prediction of time derivatives are made with a random forest (RF), a residual neural network (ResNet), and the Orthogonal Additive Kernel (OAK) inducing points scalable GP, while for the timeseries prediction comparisons are made with LSTM and GRU recurrent neural networks (RNNs) along with the SINDy package.

Hayes, Kyle↗