Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark dataset”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

NAPAbench 2: A network synthesis algorithm for generating realistic protein-protein interaction (PPI) network families

Comparative network analysis provides effective computational means for gaining novel insights into the structural and functional compositions of biological networks. In recent years, various methods have been developed for biological network alignment, whose main goal is to identify important similarities and critical differences between networks in terms of their topology and composition. A major impediment to advancing network alignment techniques has been the lack of gold-standard benchmarks that can be used for accurate and comprehensive performance assessment of such algorithms. The original NAPAbench (network alignment performance assessment benchmark) was developed to address this problem, and it has been widely utilized by many researchers for the development, evaluation, and comparison of novel network alignment techniques. In this work, we introduce NAPAbench 2—a major update of the original NAPAbench that was introduced in 2012. NAPAbench 2 includes a completely redesigned network synthesis algorithm that can generate protein-protein interaction (PPI) network families whose characteristics closely match those of the latest real PPI networks. Furthermore, the network synthesis algorithm comes with an intuitive GUI that allows users to easily generate PPI network families with an arbitrary number of networks of any size, according to a flexible user-defined phylogeny. In addition, NAPAbench 2 provides updated benchmark datasets—created using the redesigned network synthesis algorithm—which can be used for comprehensive performance assessment of network alignment algorithms and their scalability.

42 ENGINEERING↗

The two radiative states of the Arctic atmosphere and their impacts on the surface energy budget of sea ice

The surface energy budget (SEB) is a central regulator of Arctic climate and sea ice evolution, yet its processes remain poorly constrained due to sparse observations and complex, coupled surface-atmosphere interactions. This study leverages year-long measurements from the Multidisciplinary drifting Observatory for the Study of Arctic Climate (MOSAiC) to provide the most comprehensive assessment to date of the central Arctic SEB and its modulation by atmospheric variability. Ship- and ice-based observations from October 2019 to September 2020 were used to directly measure or tightly constrain each term of the SEB, leading to exceptional energetic closure with the seasonal snow and ice mass balance. The analysis reveals strong seasonal transitions in atmosphere-surface energy transfer that are modulated by the atmospheric state and constrained by the ability of the surface temperature to respond. Classification of the atmosphere into its two dominant radiative states—the semi-transparent (ST) and opaque (OP)—highlights the central role of synoptic-scale variability in clouds. The ST atmospheric state dominated the long winter ice growth season, with limited cloudiness supporting persistent surface radiative cooling and ice growth. The OP state, associated with liquid-containing or thick ice clouds, became dominant in spring, with the combination of increased solar heating and cloud surface longwave warming driving ice and snow melt. Eddy covariance versus bulk approaches for deriving surface turbulent heat fluxes provide vastly different perspectives on the role of turbulence in modulating the SEB. These results establish a high-quality benchmark dataset for Arctic SEB studies and demonstrate how the balance of atmospheric radiative states exerts a first-order control on the annual evolution of the sea ice. The findings have broad implications for advancing observing technologies, understanding Arctic amplification, improving climate models, and predicting future sea ice change.

54 ENVIRONMENTAL SCIENCES↗

Denoising Seismograms in the Time Domain Using a Deep Learning Model

Deep learning has emerged as a transformative tool for enhancing the extraction of reliable information from seismograms, addressing the increasing demand for precise and efficient seismic data analysis. We introduce an innovative encoder–decoder deep learning model, named WaveDenoiser, designed for noise reduction in the time domain, thereby eliminating the need for spectrogram computations that have been used for existing deep learning tools and significantly improving processing speed. Utilizing the benchmark dataset that is Stanford Earthquake Dataset, we developed three models of varying sizes: base, medium, and large. Notably, the large (referred to as WaveDenoiser) model demonstrated superior performance, achieving a median signal‐to‐noise ratio improvement of 8.8 dB on in‐distribution unseen data (in the same geographic region) and 7.7 dB on out‐distribution unseen data (in a new geographic region), outpacing both the base and medium models. Further evaluation of the WaveDenoiser model revealed a reduction in median arrival‐time errors by 0.02 s for P waves and 0.01 s for S waves when processing waveforms prior to phase picking using PhaseNet on in‐distribution unseen data. When tested on out‐distribution unseen data, the model also effectively reduced the P‐wave median arrival‐time error by 0.02 and 0.01 s in median arrival‐time error for S waves. Importantly, the application of WaveDenoiser resulted in a significant reduction of phase picking outliers by 1.1% to 3.6% for both P and S waves. In addition, we achieved over five times acceleration in processing speed compared with the seisBench implementation of DeepDenoiser. Our findings underscore the potential of WaveDenoiser as a powerful tool for improving seismic data analysis and processing efficiency.

P-waves↗

PickerXL, A Large Deep Learning Model to Measure Arrival Times from Noisy Seismic Signals

Precisely measuring seismic arrival times is a labor-intensive task but is critical for both earthquake monitoring and subsurface imaging. Recently published deep learning models have demonstrated superior performance compared to traditional automatic approaches for picking arrival times. Although existing deep learning models have shown promising results, further advancements are necessary as their performance is not yet satisfactory especially when applied to new regions and station networks. Increasing model size has led to improved performance in other machine learning applications. Here, we aimed to investigate whether enlarging deep learning models can increase performance on accepted benchmarks. We trained three models of varying sizes, small (1X), medium (4X), and large (16X), using globally distributed local and regional earthquake signals and background noise waveforms from a benchmark dataset, Stanford Earthquake Dataset. Our results indicate that the largest model (PickerXL) outperforms both the smaller models and Seisbench implementation of the PhaseNet model, which has the same number of parameters as our small model. The PickerXL model’s enhanced capacity to extract complex patterns from seismograms contributes to its superior arrival picking abilities compared to the smaller model.

Chai, Chengping [Oak Ridge National Laboratory (OR↗

Generative unfolding with distribution mapping

Machine learning enables unbinned, highly-differential cross section measurements. A recent idea uses generative models to morph a starting simulation into the unfolded data. We show how to extend two morphing techniques, Schrödinger Bridges and Direct Diffusion, in order to ensure that the models learn the correct conditional probabilities. This brings distribution mapping (DM) to a similar level of accuracy as the state-of-the-art conditional generative unfolding methods. Numerical results are presented with a standard benchmark dataset of single jet substructure as well as for a new dataset describing a 22-dimensional phase space of Z+2 -jets.

Butter, Anja↗

Effects of Jacobian Matrix Regularization on the Detectability of Adversarial Samples

The well-known vulnerability of Deep Neural Networks to adversarial samples has led to a rapid cycle of increasingly sophisticated attack algorithms and proposed defenses. While most contemporary defenses have been shown to be vulnerable to carefully configured attacks, methods based on gradient regularization and out-of-distribution detection have attracted much interest recently by demonstrating higher resilience to a broad range of attack algorithms. However, no study has yet investigated the effect of combining these techniques. In this paper, we consider the effect of Jacobian matrix regularization on the detectability of adversarial samples on the CIFAR-10 image benchmark dataset. We find that regularization has a significant effect on detectability, and in some cases can make an undetectable attack on a baseline model detectable. In addition, we give evidence that regularization may mitigate the known weaknesses of detectors to high-confidence adversarial samples. The defenses we consider here are highly generalizable, and we believe they will be useful for further investigations to transfer machine learning robustness to other data domains.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Deep Analysis Net with Causal Embedding for Coal-fired Power Plant Fault Detection and Diagnosis (DANCE4CFDD)

Fault detection and diagnosis is critical to power plant operation to ensure attaining high reliability while reducing operation cost. As more renewable power is introduced to the power grid, traditional fossil power plants take on the extra burden of excessive load cycling to compensate the generation variability from renewable power. Such load cycling will pose more reliability challenges to power plant operation. There are a number of challenges faced by today’s asset health management system in coal- fired (or gas) power plants: 1) high-dimensional nonlinear interaction among multiple time series measurements; 2) high measurement variance induced by operational conditions/modes; 3) variation among asset types and plant configurations; and 4) a small number of faulty events to learn from. To cope with these challenges, today’s fielded asset health management systems rely heavily on manual efforts from domain experts and hand-crafted features or rules based on domain knowledge. Despite its role in plant reliability, such a practice is costly and hinders its scalability and sustainability, particularly when a plant undergoes modifications. The objective of this project is to develop a novel end-to-end AI learning system that is trainable (i.e., the AI representation of a complex system behavior can be directly learned from properly labeled data) for accurate fault detection and root cause analysis. The ability to create a fault detection model directly from time series could alleviate the efforts associated with today’s asset management solution development. In the course of this project, we have achieved the following: Created an AI model development environment incorporating state-of-the-art neural network architectures for rapid model development and evaluation; Developed novel learning strategies for training of fault detection model; Developed special-purpose neural network architecture embedded with variable association graph aiming for better interpretability; Developed a learning strategy to leverage a small number of faulty events for enhanced fault detection capability; Conducted detailed experimental study based on public benchmark datasets and demonstrated the effectiveness of the proposed solution; and Validated the developed system with data from both a coal-fired plant boiler dynamic simulation model and real-world coal-fired power plant covering multiple asset and fault types. Overall, the project attained a technology readiness level of TRL 5 from TRL 2 at the beginning of the project.

20 FOSSIL-FUELED POWER PLANTS↗

Science Area 1: Standard Award: Model-Data Fusion to Examine Multiscale Dynamical Controls on Snow Cover and Critical Zone Moisture Inputs (Final Report)

In many mountain watersheds of the world, seasonal snowpacks play an important role as natural reservoirs of water. Seasonal snowpacks accumulate water during cold, wet winter months that subsequently melts. Downstream communities depend on water from melting seasonal snowpacks to support agricultural, industrial, and municipal water needs. Rapidly melting snowpacks can also present a flooding hazard, particularly if snowpacks melt at rates faster than anticipated and where adequate reservoir capacity is unavailable to buffer river flows associated with melt. The spatial and temporal dynamics of snow accumulation and melt also play an important role in supporting upland ecosystems in mountain landscapes. Snowmelt provides soil moisture that enable terrestrial ecosystem productivity and exert control on soil microorganisms that play important roles in global carbon cycles. Climate warming is gradually decreasing the amount of precipitation in mountain watersheds arriving as snow, presenting potentially profound disruptions to mountain ecosystems, as well as downstream delivery of water. The overarching goal of this project was to understand how interactions between the near-surface atmosphere and surface topography control the input, accumulation, retention, and release of water from mountain snowpacks. Over a 5-year period, this project pursued an approach combining high-resolution regional climate modeling, satellite and airborne remote sensing data, and ground-based observations to develop and analyze benchmark datasets to address overarching science questions and hypotheses. Key products include a continuous, long-term, high spatiotemporal resolution (1 km/1 hr) dataset characterizing key climate variables in the Upper Colorado River Basin. The dataset included historical estimates of precipitation, temperature, humidity, solar and longwave radiation, and wind speeds. Additionally, the project developed a 20+ year long, 30 m spatial, daily temporal multi-sensor dataset characterizing snow presence/absence in the East/Taylor River watersheds in the Central Rocky Mountains of Colorado. The project supported training of 1 postdoctoral scholar, 1 Ph.D. student, and 1 M.S. student.

54 ENVIRONMENTAL SCIENCES↗

Quarknet Project: Simulating Double-Source Plane Lenses and Einstein Rings for Domain Adaptation

Double-Source Plane Lenses (DSPLs) are a form of strong gravitational lensing that can be used to uniquely constrain cosmological parameters like the dark energy equation of state. These objects are exceedingly rare: to date, only four have been discovered in astronomical surveys, like the Sloan Digital Sky Survey and the HyperSuprimeCam Survey. Additionally, the main features of these objects are typically co-located sets of arcs, which look like Einstein Rings of single-source galaxy-galaxy lenses. Larger astronomical surveys typically have low image resolution, which significantly exacerbates issues when trying to find DSPLs. Future surveys are likely to include many more of these objects and have higher image resolution. Traditional methods for finding strong lenses struggle to identify simple galaxy-galaxy lenses. Recently, neural networks have become the standard for lens identification, but these tools have not yet been used to search for DSPLs. In this work, we use the software deeplenstronomy to simulate varied images of DSPLs and Einstein Rings – with and without noise and artifacts of observational surveys, forming a comprehensive benchmark dataset of 40,000 images. We utilise this dataset to perform experiments with convolutional neural networks to discern between the two phenomena. Sofia Grimm and Jerry Zhou are co-first authors on this work.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Model Card for WaveDenoiser

This study used STEAD to train and evaluate this model because STEAD is among the best benchmark datasets available for local to regional data. STEAD is a global dataset with over 1 million 60 s long seismic waveforms that originated from approximately 450,000 earthquakes and background noise captured by more than 2,500 seismic stations. Each waveform in STEAD was attached with metadata such as earthquake locations, station locations, and signal arrival times when available

58 GEOSCIENCES↗

Uncertainty Quantification for Smooth Functional Data with Application to Material Properties

This document outlines a method for processing functional output (i.e., curves) for the ultimate purpose of sampling curves under specified input conditions for use in modeling and simulation uncertainty quantification (UQ) studies. A set of benchmark curves sufficiently representative of the relevant scenario(s) being simulated are provided to the process and formatted as described in Section 1. Principal Component Analysis (PCA) is utilized to discover the components of uncertainty in the benchmark curves and is outlined in Section 2. Section 3 describes the application of uncertainty quantification to the PCA results for the purpose of sampling curves to be used in UQ analysis. Section 4 applies these techniques to an example benchmark dataset. Concluding remarks are provided in the final section.

36 MATERIALS SCIENCE↗

FAIR Data and Interpretable AI Framework for Architectured Metamaterials (Final Report)

This research program established a transformative framework for the discovery and design of mechanical metamaterials, which are architected structures engineered to control physical phenomena like sound and vibration in ways natural materials cannot. To overcome the traditional reliance on trial-and-error, the project developed an interpretable Artificial Intelligence (AI) framework that moves beyond "black box" models to reveal the specific geometric patterns—such as "unit-cell templates"—that govern a material’s performance. A major breakthrough was the development of a hierarchical design method, which allows a single material to block vibrations across multiple frequency ranges simultaneously by layering patterns at different scales without them interfering with one another. This was further expanded to include irregular, graph-based designs that use spanning tree algorithms to ensure structural connectivity while allowing for customized, direction-dependent properties like stiffness and acoustic impedance. Beyond design, the project addressed the practicalities of real-world production by developing uncertainty quantification techniques that account for manufacturing defects and material variability, reducing the need for expensive physical testing by orders of magnitude. To speed up the discovery process, the team implemented Gaussian Process Regression and other surrogate models that provide accurate performance predictions at a fraction of the traditional computational cost. The AI-generated designs were successfully validated through fabrication of physical samples and wave propagation experiments, confirming their ability to accurately guide or reflect waves as predicted. By contributing these tools and high-quality FAIR benchmark datasets to the wider scientific community, this work provides a scalable foundation for advancing technologies in aerospace vibration control, medical imaging, and noise reduction.

36 MATERIALS SCIENCE↗

Establishment of a Vertically Integrated Domestic Manufacturing Process for Production of Substrates Needed for Manufacture of Gas Diffusion Layers

In this project, AvCarb, LLC evaluated the baseline performance metrics of commercial carbon veils and their corresponding Gas Diffusion Layers (GDLs) with the goal of establishing an optimized, vertically integrated production system for wet-laid nonwoven substrates used in gas diffusion media for electrochemical energy storage and conversion devices. Mechanical testing and microstructural characterization were conducted and used to develop a multiscale computational model capable of simulating and predicting the performance of GDLs in fuel cells. Although the project successfully generated foundational transport and modeling data, it was terminated prior to identifying the critical GDL design parameters necessary for full optimization. The program aimed to improve carbon veil fabrication through enhanced fiber dispersion, fiber-fiber adhesion control, and improved web formation, enabling the production of high-quality, uniform substrates. Simulations were intended to guide mixing and solution delivery system design and process conditions, followed by production-scale trials to evaluate fiber dispersion, web uniformity, and mechanical robustness. At full deployment, the proposed production line would have been capable of producing approximately 650,000 m² of carbon veil annually. This capability remains strategically important, as the United States currently lacks a domestic source of wet-laid nonwoven carbon substrates that satisfy the stringent quality requirements for fuel-cell GDLs and electrolyzers representing an ongoing supply-chain vulnerability. Beyond supply-chain benefits, the project established a robust benchmarking dataset for existing commercial carbon veils while advancing next-generation material concepts targeting improved performance and manufacturing consistency.

Olson, Cynthia Lemay↗

FAIR Data and Interpretable AI Framework for Architectured Metamaterials

Our interdisciplinary effort successfully generated FAIR (Findable, Accessible, Interoperable, and Reusable) benchmark datasets for mechanical metamaterials while introducing a novel Artificial Intelligence (AI) framework known as Learning Refined Compositional Rules (LRCR). This framework was specifically designed to bridge the gap across varying computational length scales and extract the underlying physical mechanisms that connect a material's structural geometry to its bulk acoustic properties. Historically, the discovery of such structured materials relied heavily on human intuition or opaque, black-box optimization algorithms that were difficult to generalize. By combining interpretable machine learning techniques with rigorous experimental validation, this project established clear, generalizable design guidelines for tuning wave dispersion and controlling vibrations. Ultimately, the public availability of these structured datasets and algorithms will significantly reduce computational costs and accelerate the design of advanced multi-functional acoustic devices, offering broad societal impacts across fields like aerospace engineering, telecommunications, and biomedical implant design.

36 MATERIALS SCIENCE↗

Physics-Informed Deep Learning for Reconstruction of Spatial Missing Climate Information in the Antarctic

Understanding the influence of the Antarctic on the global climate is crucial for the prediction of global warming. However, due to very few observation sites, it is difficult to reconstruct the rational spatial pattern by filling in the missing values from the limited site observations. To tackle this challenge, regional spatial gap-filling methods, such as Kriging and inverse distance weighted (IDW), are regularly used in geoscience. Nevertheless, the reconstructing credibility of these methods is undesirable when the spatial structure has massive missing pieces. Inspired by image inpainting, we propose a novel deep learning method that demonstrates a good effect by embedding the physics-aware initialization of deep learning methods for rapid learning and capturing the spatial dependence for the high-fidelity imputation of missing areas. We create the benchmark dataset that artificially masks the Antarctic region with ratios of 30%, 50% and 70%. The reconstructing monthly mean surface temperature using the deep learning image inpainting method RFR (Recurrent Feature Reasoning) exhibits an average of 63% and 71% improvement of accuracy over Kriging and IDW under different missing rates. With regard to wind speed, there are still 36% and 50% improvements. In particular, the achieved improvement is even better for the larger missing ratio, such as under the 70% missing rate, where the accuracy of RFR is 68% and 74% higher than Kriging and IDW for temperature and also 38% and 46% higher for wind speed. In addition, the PI-RFR (Physics-Informed Recurrent Feature Reasoning) method we proposed is initialized using the spatial pattern data simulated by the numerical climate model instead of the unified average. Compared with RFR, PI-RFR has an average accuracy improvement of 10% for temperature and 9% for wind speed. When applied to reconstruct the spatial pattern based on the Antarctic site observations, where the missing rate is over 90%, the proposed method exhibits more spatial characteristics than Kriging and IDW.

54 ENVIRONMENTAL SCIENCES↗

Development of inter-grid-cell lateral unsaturated and saturated flow model in the E3SM Land Model (v2.0)

Abstract. The lateral transport of water in the subsurface is important in modulating terrestrial water energy distribution. Although a few land surface models have recently included lateral saturated flow within and across grid cells, it is not a default configuration in the Climate Model Intercomparison Project version 6 experiments. In this work, we developed the lateral subsurface flow model within both unsaturated and saturated zones in the Energy Exascale Earth System Model (E3SM) Land Model version 2 (ELMv2.0). The new model, called ELMlat, was benchmarked against PFLOTRAN, a 3D subsurface flow and transport model, for three idealized hillslopes that included a convergent hillslope, divergent hillslope, and tilted V-shaped hillslope with variably saturated initial conditions. ELMlat showed comparable performance against PFLOTRAN in terms of capturing the dynamics of soil moisture and groundwater table for the three benchmark hillslope problems. Specifically, the mean absolute errors (MAEs) of the soil moisture in the top 10 layers between ELMlat and PFLOTRAN were within 1 %±3 %, and the MAEs of water table depth were within ±0.2 m. Next, ELMlat was applied to the Little Washita experimental watershed to assess its prediction of groundwater table, soil moisture, and soil temperature. The spatial pattern of simulated groundwater table depth agreed well with the global groundwater table benchmark dataset generated from a global model calibrated with long-term observations. The effects of lateral groundwater flow on the energy flux partitioning were more prominent in lowland areas with shallower groundwater tables, where the difference in simulated annual surface soil temperature could reach 0.3–0.4 ∘C between ELMv2.0 and ELMlat. Incorporating lateral subsurface flow in ELM improves the representation of the subsurface hydrology, which will provide a good basis for future large-scale applications.

58 GEOSCIENCES↗

Evaluation of E3SM land model snow simulations over the western United States

Abstract. Seasonal snow has crucial impacts on climate, ecosystems, and humans, but it is vulnerable to global warming. The land component (ELM) of the Energy Exascale Earth System Model (E3SM) mechanistically simulates snow processes from accumulation, canopy interception, compaction, and snow aging to melt. Although high-quality field measurements, remote sensing snow products, and data assimilation products with high spatio-temporal resolution are available, there has been no systematic evaluation of the snow properties and phenology in ELM. This study comprehensively evaluates ELM snow simulations over the western United States at 0.125∘ resolution during 2001–2019 using the Snow Telemetry (SNOTEL) in situ networks, MODIS remote sensing products (i.e., MCD43 surface albedo product), the spatially and temporally complete (STC) snow-covered area and grain size (MODSCAG) and MODIS dust and radiative forcing in snow (MODDRFS) products (STC-MODSCAG/STC-MODDRFS), and the snow property inversion from remote sensing (SPIReS) product and two data assimilation products of snow water equivalent and snow depth – i.e., University of Arizona (UA) and SNOw Data Assimilation System (SNODAS). Overall the ELM simulations are consistent with the benchmarking datasets and reproduce the spatio-temporal patterns, interannual variability, and elevation gradients for different snow properties including snow cover fraction (fsno), surface albedo (αsur) over snow cover regions, snow water equivalent (SWE), and snow depth (Dsno). However, there are large biases of fsno with dense forest cover and αsur in the Rocky Mountains and Sierra Nevada in winter, compared to the MODIS products. There are large discrepancies of snow albedo, snow grain size, and light-absorbing particle-induced snow albedo reduction between ELM and the MODIS products, attributed to uncertainties in the aerosol forcing data, snow aging processes in ELM, and remote sensing retrievals. Against UA and SNODAS, ELM has a mean bias of −20.7 mm (−35.9 %) and −20.4 mm (−35.5 %), respectively, for spring, and −13.8 mm (−27.8 %) and −10.2 mm (−22.2 %), respectively, for winter. ELM shows a relatively high correlation with SNOTEL SWE, with mean correlation coefficients of 0.69 but negative mean biases of −122.7 mm. Compared to the snow phenology of STC-MODSCAG and SPIReS, ELM shows delayed snow accumulation onset dates by 17.3 and 12.4 d, earlier snow end dates by 35.5 and 26.8 d, and shorter snow durations by 52.9 and 39.5 d, respectively. This study underscores the need for diagnosing model biases and improving ELM representations of snow properties and snow phenology in mountainous areas for more credible simulation and future projection of mountain snowpack.

54 ENVIRONMENTAL SCIENCES↗

Anytime query-tuned kernel machine classifiers via Cholesky factorization

We recently demonstrated 2 to 64-fold query-time speedups of Support Vector Machine and Kernel Fisher classifiers via a new computational geometry method for anytime output bounds (DeCoste,2002). This new paper refines our approach in two key ways. First, we introduce a simple linear algebra formulation based on Cholesky factorization, yielding simpler equations and lower computational overhead. Second, this new formulation suggests new methods for achieving additional speedups, including tuning on query samples. We demonstrate effectiveness on benchmark datasets.

Kernel Cholesky MNIST↗