Search NASA⌕ Search

SEARCH · Search NASA

Results for “evaluation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43

Autonomous Selective Parts-Based Tracking

Object tracking from videos is still a challenging task due to various changes throughout a video sequence including occlusions, motion blur, scale and other deformation changes. In this paper, we propose a selective parts-based approach, using correlation filters, that makes choices based on a consensus of the parts and global tracking. Moreover, we further enhance our parts-based approach by introducing a segmentation-assisted parts initialization. In addition, we present a genetic algorithm-based method to autonomously select various parameters of the tracking algorithm, as opposed to the common practice of manually tuning those parameters. In contrast to existing part-based methods, the proposed method does not dilute accurate tracking by averaging results over multiple parts at every frame. Instead, we take a selective approach based on the relative weight of the responses across parts. Moreover, we only make location corrections when a part diverges, and rely on these location corrections to maintain an accurate appearance model. In the case of occlusions, which are among the main reasons for using a parts-based approach, our proposed approach consistently achieves the best performance. It is due to the ability to handle occlusion and not dilute decisions with incorrect parts, that our proposed approach enables state-of-the-art performance. The approach we propose was evaluated on videos from three different challenging benchmark datasets. Our approach has resulted in better overall precision and success rates for three different base tracking approaches.

42 ENGINEERING↗

A Data-Driven Framework for Power System Event Type Identification via Safe Semi-Supervised Techniques

Herein this paper investigates the use of phasor measurement unit (PMU) data with deep learning techniques to construct real-time event identification models for transmission networks. Increasing penetration of distributed energy resources represents a great opportunity to achieve decarbonization, as well as challenges in systematic situational awareness. When high-resolution PMU data and sufficient manually recorded event labels are available, the power event identification problem is defined as a statistical classification problem that can be solved by numerous cutting-edge classifiers. However, in real grids, collecting tremendous high-quality event labels is quite expensive. Utilities frequently have a large number of event records without in-depth details (i.e., unlabeled events). To bridge this gap, we propose a novel semi-supervised learning-based method to improve the performance of event classifiers trained with a limited number of labeled events by exploiting the information from massive unlabeled events. In other words, compared to existing data-driven methods, our method requires only a small portion of labeled data to achieve a similar level of accuracy. Meanwhile, this work discusses and addresses the performance degradation caused by class distribution mismatch between the training set and the real applications. Based on the proposed safe learning mechanism, our model does not directly use all unlabeled events during model training, but selectively uses them through a comprehensive evaluation procedure. Numerical studies on a sizable PMU dataset have been used to validate the performance of the proposed method.

42 ENGINEERING↗

SigTime: Learning and Visually Explaining Time Series Signatures

Understanding and distinguishing temporal patterns in time series data is essential for scientific discovery and decision-making. For example, in biomedical research, uncovering meaningful patterns in physiological signals can improve diagnosis, risk assessment, and patient outcomes. However, existing methods for time series pattern discovery face major challenges, including high computational complexity, limited interpretability, and difficulty in capturing meaningful temporal structures. Here, to address these gaps, we introduce a novel learning framework that jointly trains two Transformer models using complementary time series representations: shapelet-based representations to capture localized temporal structures and traditional feature engineering to encode statistical properties. The learned shapelets serve as interpretable signatures that differentiate time series across classification labels. Additionally, we develop a visual analytics system—SigTime—with coordinated views to facilitate exploration of time series signatures from multiple perspectives, aiding in useful insights generation. We quantitatively evaluate our learning framework on eight publicly available datasets and one proprietary clinical dataset. Additionally, we demonstrate the effectiveness of our system through two usage scenarios along with the domain experts: one involving public ECG data and the other focused on preterm labor analysis.

97 MATHEMATICS AND COMPUTING↗

Multiattribute probabilistic prostate elastic registration (MAPPER): Application to fusion of ultrasound and magnetic resonance imaging

Purpose: Transrectal ultrasound (TRUS)-guided needle biopsy is the current gold standard for prostate cancer diagnosis. However, up to 40% of prostate cancer lesions appears isoechoic on TRUS. Hence, TRUS-guided biopsy has a high false negative rate for prostate cancer diagnosis. Magnetic resonance imaging (MRI) is better able to distinguish prostate cancer from benign tissue. However, MRI-guided biopsy requires special equipment and training and a longer procedure time. MRI-TRUS fusion, where MRI is acquired preoperatively and then aligned to TRUS, allows for advantages of both modalities to be leveraged during biopsy. MRI-TRUS-guided biopsy increases the yield of cancer positive biopsies. In this work, the authors present multiattribute probabilistic postate elastic registration (MAPPER) to align prostate MRI and TRUS imagery. Methods: MAPPER involves (1) segmenting the prostate on MRI, (2) calculating a multiattribute probabilistic map of prostate location on TRUS, and (3) maximizing overlap between the prostate segmentation on MRI and the multiattribute probabilistic map on TRUS, thereby driving registration of MRI onto TRUS. MAPPER represents a significant advancement over the current state-of-the-art as it requires no user interaction during the biopsy procedure by leveraging texture and spatial information to determine the prostate location on TRUS. Although MAPPER requires manual interaction to segment the prostate on MRI, this step is performed prior to biopsy and will not substantially increase biopsy procedure time. Results: MAPPER was evaluated on 13 patient studies from two independent datasets—Dataset 1 has 6 studies acquired with a side-firing TRUS probe and a 1.5 T pelvic phased-array coil MRI; Dataset 2 has 7 studies acquired with a volumetric end-firing TRUS probe and a 3.0 T endorectal coil MRI. MAPPER has a root-mean-square error (RMSE) for expert selected fiducials of 3.36 ± 1.10 mm for Dataset 1 and 3.14 ± 0.75 mm for Dataset 2. State-of-the-art MRI-TRUS fusion methods report RMSE of 3.06–2.07 mm. Conclusions: MAPPER aligns MRI and TRUS imagery without manual intervention ensuring efficient, reproducible registration. MAPPER has a similar RMSE to state-of-the-art methods that require manual intervention.

60 APPLIED LIFE SCIENCES↗

Simulation-Based Inference for Neutrino Parameter Tuning

This code trains a simulation-based inference (SBI) model for neutrino interaction parameter tuning and subsequently evaluates its performance on MicroBooNE, T2K, and NuWro datasets.

Tame-Narvaez, Karla [Fermi National Accelerator La↗

Improving Mixed-phase Cloud Parameterization in Climate Model with the ACRF Measurements

Mixed-phase cloud microphysical and dynamical processes are still poorly understood, and their representation in GCMs is a major source of uncertainties in overall cloud feedback in GCMs. Thus improving mixed-phase cloud parameterizations in climate models is critical to reducing the climate forecast uncertainties. This study aims at providing improved knowledge of mixed-phase cloud properties from the long-term ACRF observations and improving mixed-phase clouds simulations in the NCAR Community Atmosphere Model version 5 (CAM5). The key accomplishments are: 1) An improved retrieval algorithm was developed to provide liquid droplet concentration for drizzling or mixed-phase stratiform clouds. 2) A new ice concentration retrieval algorithm for stratiform mixed-phase clouds was developed. 3) A strong seasonal aerosol impact on ice generation in Arctic mixed-phase clouds was identified, which is mainly attributed to the high dust occurrence during the spring season. 4) A suite of multi-senor algorithms was applied to long-term ARM observations at the Barrow site to provide a complete dataset (LWC and effective radius profile for liquid phase, and IWC, Dge profiles and ice concentration for ice phase) to characterize Arctic stratiform mixed-phase clouds. This multi-year stratiform mixed-phase cloud dataset provides necessary information to study related processes, evaluate model stratiform mixed-phase cloud simulations, and improve model stratiform mixed-phase cloud parameterization. 5). A new in situ data analysis method was developed to quantify liquid mass partition in convective mixed-phase clouds. For the first time, we reliably compared liquid mass partitions in stratiform and convective mixed-phase clouds. Due to the different dynamics in stratiform and convective mixed-phase clouds, the temperature dependencies of liquid mass partitions are significantly different due to much higher ice concentrations in convective mixed phase clouds. 6) Systematic evaluations of mixed-phase cloud simulations by CAM5 were performed. Measurement results indicate that ice concentrations control stratiform mixed-phase cloud properties. The improvement of ice concentration parameterization in the CAM5 was done in close collaboration with Dr. Xiaohong Liu, PNNL (now at University of Wyoming).

54 ENVIRONMENTAL SCIENCES↗

Synthetic data generators for the evaluation of biosurveillance outbreak detection algorithms

The research and development of new algorithmic and statistical methods of outbreak detection is an ongoing research priority in the field of biosurveillance. The early detection of emergent disease outbreaks is crucial for effective treatment and mitigation. New detection methods must be compared to established approaches for proper evaluation. This comparison requires biosurveillance test data that accurately reflects the complexity of the real-world data it will be applied to. While the test and evaluation of new detection methods is best performed on real data, it is often impractical to obtain such data as it is either proprietary or limited in scope. Thus, scientists must turn to synthetic data generation to provide enough data to properly eval- uate new detection methodologies. This paper evaluates three such synthetic data sources: The WSARE dataset, the Noufilay equation-based approach, and the Project Mimic data generator.

47 OTHER INSTRUMENTATION↗

Improving Predictability of Mixed-Phase Clouds and Aerosol Interactions in the Community Earth System Model (CESM) with ARM Measurements

The objective of this project is to improve the simulation and predictability of mixed-phase clouds and aerosol interactions in the Community Earth System Model (CESM) through comparisons with the DOE ARM observations. There are four major goals of the proposed study: (1) Improve the representation of ice microphysical processes in mixed-phase clouds; (2) Develop a long-term multi-sensor mixed-phase cloud dataset; (3) Test the performance of improved ice microphysics in CESM-CAM5 with the ARM data; and (4) Examine mixed-phase cloud microphysics-aerosol-dynamics-radiation interactions in CESM-CAM5. In this project, we have (1) Improved the representation of ice microphysical processes in mixed-phase clouds that include ice nucleation, ice depositional growth through the Wegener–Bergeron–Findeisen (WBF) process, feedbacks between cloud microphysics, dynamics and radiation, and provides links to aerosol properties in the Community Earth System Model (CESM)-Community Atmosphere Model version 5 (CAM5); (2) Developed a long-term multi-sensor mixed-phase cloud dataset. The mixed-phase cloud retrieval algorithms are improved by refining the phase determinations and ice number concentration retrievals. A multi-year mixed-phase cloud observation dataset (1999-2004; October 2013-February 2017) is regenerated at the ARM Barrow site with the improved algorithms. The dataset has been used in the global model evaluation to improve the model parameterizations; (3) Tested the performance of improved ice microphysics in CESM-CAM5 with the ARM data. CESM-CAM5 is run in the DOE Cloud-Associated Parameterizations Testbed (CAPT) and in the single-column model (SCM) mode to facilitate comparison with ARM observations. Multi-year CAPT simulations of seasonal variations of cloud microphysical properties, cloud liquid water path (LWP) and ice water path (IWP), cloud longwave and shortwave forcings, cloud occurrence frequency are evaluated against the ARM multi-sensor mixed-phase cloud retrievals; and (4) Examined mixed-phase cloud microphysics-aerosol-dynamics-radiation interactions in CESM-CAM5 that include the impacts of different ice nucleation mechanisms, sensitivity to dust IN concentration, treatment of the WBF process on mixed-phase cloud properties. Aerosol effect on mixed-phase clouds through the liquid phase (droplet activation) and ice phase processes (e.g., the glaciation indirect effect) is investigated.

54 ENVIRONMENTAL SCIENCES↗

Surface and Buried Thermal, and RGB Unexploded Ordnance Data Collection

This document provides a description of a data collection campaign of unexploded ordnance (UXOI) set. The dataset captures a controlled UAV imaging campaign designed to support detection of UXO across varied environmental conditions. Data were collected during three campaigns in Norris and Northeast Knoxville, Tennessee, using RGB, and thermal sensors mounted on Parrot UKR. In total, the dataset contains 9925 images, 26 full-motion video, and approximately 81.99 GB of data, collected across late spring/summer conditions, every hour during sunlight, and multiple surface contexts, including tall grass, short grass, gravel, as well as buried in sand, and other gravel mixtures. The collection was designed to capture thermal and visual variability relevant to UXO detection in agricultural land, bare earth, and subsurface. Review of the imagery showed that ordnance was most detectable during periods of changing solar input, especially approximately 10-60 minutes after sunrise, approximately 20-60 minutes after sunset, and 2-3 min after cloud cover interrupted prolonged solar heating. These conditions increased thermal contrast because many ordnance items retained or released heat differently than the surrounding vegetation and ground surface. This dataset provides a useful resource for developing and evaluating airborne UXO detection methods under realistic field conditions. All ordnance used in the study was inert, and thermal behavior may differ from that of live ordnance. In addition, variation in ordnance type, composition, and placement introduced differences in thermal response that should be considered when interpreting results.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

The Clinical Assessment and Remote Administration Tablet

Electronic data capture of case report forms, demographic, neuropsychiatric, or clinical assessments, can vary from scanning hand-written forms into databases to fully electronic systems. Web-based forms can be extremely useful for self-assessment; however, in the case of neuropsychiatric assessments, self-assessment is often not an option. The clinician often must be the person either summarizing or making their best judgment about the subject’s response in order to complete an assessment, and having the clinician turn away to type into a web browser may be disruptive to the flow of the interview. The Mind Research Network has developed a prototype for a software tool for the real-time acquisition and validation of clinical assessments in remote environments.We have developed the clinical assessment and remote administration tablet on a Microsoft Windows PC tablet system, which has been adapted to interact with various data models already in use in several large-scale databases of neuroimaging studies in clinical populations. The tablet has been used successfully to collect and administer clinical assessments in several largescale studies, so that the correct clinical measures are integrated with the correct imaging and other data. It has proven to be incredibly valuable in confirming that data collection across multiple research groups is performed similarly, quickly, and with accountability for incomplete datasets. We present the overall architecture and an evaluation of its use.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Characterizing Families of Spectral Similarity Scores and Their Use Cases for Gas Chromatography–Mass Spectrometry Small Molecule Identification

Metabolomics provides a unique snapshot into the world of small molecules and the complex biological processes that govern the human, animal, plant, and environmental ecosystems encapsulated by the One Health modeling framework. However, this “molecular snapshot” is only as informative as the number of metabolites confidently identified within it. The spectral similarity (SS) score is traditionally used to identify compound(s) in mass spectrometry approaches to metabolomics, where spectra are matched to reference libraries of candidate spectra. Unfortunately, there is little consensus on which of the dozens of available SS metrics should be used. This lack of standard SS score creates analytic uncertainty and potentially leads to issues in reproducibility, especially as these data are integrated across other domains. In this work, we use metabolomic spectral similarity as a case study to showcase the challenges in consistency within just one piece of the One Health framework that must be addressed to enable data science approaches for One Health problems. Here, using a large cohort of datasets comprising both standard and complex datasets with expert-verified truth annotations, we evaluated the effectiveness of 66 similarity metrics to delineate between correct matches (true positives) and incorrect matches (true negatives). We additionally characterize the families of these metrics to make informed recommendations for their use. Our results indicate that specific families of metrics (the Inner Product, Correlative, and Intersection families of scores) tend to perform better than others, with no single similarity metric performing optimally for all queried spectra. This work and its findings provide an empirically-based resource for researchers to use in their selection of similarity metrics for GC-MS identification, increasing scientific reproducibility through taking steps towards standardizing identification workflows.

59 BASIC BIOLOGICAL SCIENCES↗

Advancing Tassel Detection and Counting: Annotation and Algorithms

Tassel counts provide valuable information related to flowering and yield prediction in maize, but are expensive and time-consuming to acquire via traditional manual approaches. High-resolution RGB imagery acquired by unmanned aerial vehicles (UAVs), coupled with advanced machine learning approaches, including deep learning (DL), provides a new capability for monitoring flowering. In this article, three state-of-the-art DL techniques, CenterNet based on point annotation, task-aware spatial disentanglement (TSD), and detecting objects with recursive feature pyramids and switchable atrous convolution (DetectoRS) based on bounding box annotation, are modified to improve their performance for this application and evaluated for tassel detection relative to Tasselnetv2+. The dataset for the experiments is comprised of RGB images of maize tassels from plant breeding experiments, which vary in size, complexity, and overlap. Results show that the point annotations are more accurate and simpler to acquire than the bounding boxes, and bounding box-based approaches are more sensitive to the size of the bounding boxes and background than point-based approaches. Overall, CenterNet has high accuracy in comparison to the other techniques, but DetectoRS can better detect early-stage tassels. The results for these experiments were more robust than Tasselnetv2+, which is sensitive to the number of tassels in the image.

54 ENVIRONMENTAL SCIENCES↗

Dataset, Code, and Models for Training Deep Learning Potentials for Low Temperature Plasma-Surface Interactions

This repository contains datasets, training scripts, and finished models, and test simulations used in the development of DeepREBO— a machine-learned interatomic potential trained to emulate the REBO2 empirical potential. The data was generated to study deep potential development for simulations of plasma-surface interactions. It uses an active learning framework, starting from a minimal dataset and iteratively expanding it. Included are those generated datasets, the trained models, and simulations used to evaluate the performance of the training process. This resource supports reproducibility and provides a reference framework for training deep potentials in plasma-surface interaction studies.

active learning↗

Inverse modeling of CO2 sources and sinks using satellite observations of CO2 from TES and surface flask measurements

We infer CO2 surface fluxes using satellite observations of mid-tropospheric CO2 from the Tropospheric Emission Spectrometer (TES) and measurements of CO2 from surface flasks in a time-independent inversion analysis based on the GEOS-Chem model. Using TES CO2 observations over oceans, spanning 40 S 40 N, we find that the horizontal and vertical coverage of the TES and flask data are complementary. This complementarity is demonstrated by combining the datasets in a joint inversion, which provides better constraints than from either dataset alone, when a posteriori CO2 distributions are evaluated against independent ship and aircraft CO2 data. In particular, the joint inversion offers improved constraints in the tropics where surface measurements are sparse, such as the tropical forests of South America. Aggregating the annual surface-to-atmosphere fluxes from the joint inversion for the year 2006 yields 1.13 0.21 PgC for the global ocean, 2.77 0.20 PgC for the global land biosphere and 3.90 0.29 PgC for the total global natural flux (defined as the sum of all biospheric, oceanic, and biomass burning contributions but excluding CO2 emissions from fossil fuel combustion). These global ocean and global land fluxes are shown to be near the median of the broad range of values from other inversion results for 2006. To achieve these results, a bias in TES CO2 in the Southern Hemisphere was assessed and corrected using aircraft flask data, and we demonstrate that our results have low sensitivity to variations in the bias correction approach. Overall, this analysis suggests that future carbon data assimilation systems can benefit by integrating in situ and satellite observations of CO2 and that the vertical information provided by satellite observations of mid-tropospheric CO2 combined with measurements of surface CO2, provides an important additional constraint for flux inversions.

09 BIOMASS FUELS↗

Variability of Surface Temperature and Melt on the Greenland Ice Sheet, 2000-2011

Enhanced melting along with surface-temperature increases measured using infrared satellite data, have been documented for the Greenland Ice Sheet. Recently we developed a climate-quality data record of ice-surface temperature (IST) of the Greenland Ice Sheet using the Moderate-Resolution Imaging Spectroradiometer (MODIS) 1ST product -- http://modis-snow-ice.gsfc.nasa.gov. Using daily and mean monthly MODIS 1ST maps from the data record we show maximum extent of melt for the ice sheet and its six major drainage basins for a 12-year period extending from March of 2000 through December of 2011. The duration of the melt season on the ice sheet varies in different drainage basins with some basins melting progressively earlier over the study period. Some (but not all) of the basins also show a progressively-longer duration of melt. The short time of the study period (approximately 12 years) precludes an evaluation of statistically-significant trends. However the dataset provides valuable information on natural variability of IST, and on the ability of the MODIS instrument to capture changes in IST and melt conditions indifferent drainage basins of the ice sheet.

Hall, Dorothy K.↗

Global and Regional Evaluation of Over-Land Spectral Aerosol Optical Depth Retrievals from SeaWiFS

This study evaluates a new spectral aerosol optical depth (AOD) dataset derived from Sea-viewing Wide Field-of-view Sensor (Sea WiFS) measurements over land. First, the data are validated against Aerosol Robotic Network (AERONET) direct-sun AOD measurements, and found to compare well on a global basis. If only data with the highest quality flag are used, the correlation is 0.86 and 72% of matchups fall within an expected absolute uncertainty of 0.05 + 20% (for the wavelength of 550 nm). The quality is similar at other wavelengths and stable over the 13-year (1997-2010) mission length. Performance tends to be better over vegetated, low-lying terrain with typical AOD of 0.3 or less, such as found over much of North America and Eurasia. Performance tends to be poorer for low-AOD conditions near backscattering geometries, where Sea WiFS overestimates AOD, or optically-thick cases of absorbing aerosol, where SeaWiFS tends to underestimate AOD. Second, the SeaWiFS data are compared with midvisible AOD derived from the Moderate Resolution Imaging Spectrometer (MODIS) and Multiangle Imaging Spectroradiometer (MISR). All instruments show similar spatial and seasonal distributions of AOD, although there are regional and seasonal offsets between them. At locations where AERONET data are available, these offsets are largely consistent with the known validation characteristics of each dataset. With the results of this study in mind, the SeaWiFS over-land AOD record should be suitable for quantitative scientific use.

Sayer, A. M.↗

Surface Temperature and Melt on the Greenland Ice Sheet, 2000 - 2011

Enhanced melting along with surface-temperature increases measured using infrared satellite data, have been documented for the Greenland Ice Sheet. Recently we developed a climate-quality data record of ice-surface temperature (IST) of the Greenland Ice Sheet using the Moderate-Resolution Imaging Spectroradiometer (MODIS) IST product -- http://modis-snow-ice.gsfc.nasa.gov.Using daily and mean-monthly MODIS IST maps from the data record we show maximum extent of melt for the ice sheet and its six major drainage basins for a 12-year period extending from March of 2000 through December of 2011. The duration of the melt season on the ice sheet varies in different drainage basins with some basins melting progressively earlier over the study period. Some (but not all) of the basins also show a progressively-longer duration of melt. The short time of the study period (approx 12 years) precludes an evaluation of statistically-significant trends. However the dataset provides valuable information on natural variability of IST, and on the ability of the MODIS instrument to capture changes in IST and melt conditions in different drainage basins of the ice sheet.

Hall, Dorothy K.↗

The Description and Validation of a Computationally-Efficient CH4-CO-OH (ECCOH) Module for 3D Model Applications

We present the Efficient CH4-CO-OH Module (ECCOH) that allows for the simulation of the methane, carbon monoxide and hydroxyl radical (CH4-CO-OH cycle, within a chemistry climate model, carbon cycle model, or earth system model. The computational efficiency of the module allows many multi-decadal, sensitivity simulations of the CH4-CO-OH cycle, which primarily determines the global tropospheric oxidizing capacity. This capability is important for capturing the nonlinear feedbacks of the CH4-CO-OH system and understanding the perturbations to relatively long-lived methane and the concomitant impacts on climate. We implemented the ECCOH module into the NASA GEOS-5 Atmospheric Global Circulation Model (AGCM), performed multiple sensitivity simulations of the CH4-CO-OH system over two decades, and evaluated the model output with surface and satellite datasets of methane and CO. The favorable comparison of output from the ECCOH module (as configured in the GEOS-5 AGCM) with observations demonstrates the fidelity of the module for use in scientific research.

Remote Sensing↗