Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,747 records · Page 97
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate

Abstract This is a test case study assessing the ability of deep learning methods to generalize to a future climate (end of 21st century) when trained to classify thunderstorms in model output representative of the present‐day climate. A convolutional neural network (CNN) was trained to classify strongly rotating thunderstorms from a current climate created using the Weather Research and Forecasting model at high‐resolution, then evaluated against thunderstorms from a future climate and found to perform with skill and comparatively in both climates. Despite training with labels derived from a threshold value of a severe thunderstorm diagnostic (updraft helicity), which was not used as an input attribute, the CNN learned physical characteristics of organized convection and environments that are not captured by the diagnostic heuristic. Physical features were not prescribed but rather learned from the data, such as the importance of dry air at mid‐levels for intense thunderstorm development when low‐level moisture is present (i.e., convective available potential energy). Explanation techniques also revealed that thunderstorms classified as strongly rotating are associated with learned rotation signatures. Results show that the creation of synthetic data with ground truth is a viable alternative to human‐labeled data and that a CNN is able to generalize a target using learned features that would be difficult to encode due to spatial complexity. Most importantly, results from this study show that deep learning is capable of generalizing to future climate extremes and can exhibit out‐of‐sample robustness with hyperparameter tuning in certain applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

A benchmark dataset for Hydrogen Combustion

The generation of reference data for deep learning models is challenging for reactive systems, and more so for combustion reactions due to the extreme conditions that create radical species and alternative spin states during the combustion process. Here, we extend intrinsic reaction coordinate (IRC) calculations with ab initio MD simulations and normal mode displacement calculations to more extensively cover the potential energy surface for 19 reaction channels for hydrogen combustion. A total of ~290,000 potential energies and ~1,270,000 nuclear force vectors are evaluated with a high quality range-separated hybrid density functional, ωB97X-V, to construct the reference data set, including transition state ensembles, for the deep learning models to study hydrogen combustion reaction.

08 HYDROGEN↗
DOE OSTI2023

DeepPatent2: A Large-Scale Benchmarking Corpus for Technical Drawing Understanding

Abstract Recent advances in computer vision (CV) and natural language processing have been driven by exploiting big data on practical applications. However, these research fields are still limited by the sheer volume, versatility, and diversity of the available datasets. CV tasks, such as image captioning, which has primarily been carried out on natural images, still struggle to produce accurate and meaningful captions on sketched images often included in scientific and technical documents. The advancement of other tasks such as 3D reconstruction from 2D images requires larger datasets with multiple viewpoints. We introduce DeepPatent2, a large-scale dataset, providing more than 2.7 million technical drawings with 132,890 object names and 22,394 viewpoints extracted from 14 years of US design patent documents. We demonstrate the usefulness of DeepPatent2 with conceptual captioning. We further provide the potential usefulness of our dataset to facilitate other research areas such as 3D image reconstruction and image retrieval.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Complexity-calibrated benchmarks for machine learning reveal when prediction algorithms succeed and mislead

Abstract Recurrent neural networks are used to forecast time series in finance, climate, language, and from many other domains. Reservoir computers are a particularly easily trainable form of recurrent neural network. Recently, a “next-generation” reservoir computer was introduced in which the memory trace involves only a finite number of previous symbols. We explore the inherent limitations of finite-past memory traces in this intriguing proposal. A lower bound from Fano’s inequality shows that, on highly non-Markovian processes generated by large probabilistic state machines, next-generation reservoir computers with reasonably long memory traces have an error probability that is at least $$\sim 60\%$$ ∼ 60 % higher than the minimal attainable error probability in predicting the next observation. More generally, it appears that popular recurrent neural networks fall far short of optimally predicting such complex processes. These results highlight the need for a new generation of optimized recurrent neural network architectures. Alongside this finding, we present concentration-of-measure results for randomly-generated but complex processes. One conclusion is that large probabilistic state machines—specifically, large $$\epsilon$$ ϵ -machines—are key to generating challenging and structurally-unbiased stimuli for ground-truthing recurrent neural network architectures.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Benchmarking greenhouse gas emissions from US wastewater treatment for targeted reduction

Here, in this study, to assess the national climate impact of wastewater treatment and inform decarbonization, we assembled a comprehensive greenhouse gas inventory of 15,863 facilities in the contiguous USA. Considering location and treatment configurations, we modelled on-site CH 4 , N 2 O and CO 2 production and emissions associated with energy, chemical inputs and solids disposal. Using Monte Carlo simulations, we estimated median national emissions at 47 million tonnes of CO 2 equivalent per year, with on-site process CH 4 and N 2 O emissions exceeding current government estimates by 41%. Treatment configurations with anaerobic digesters are responsible for 16 million tonnes of CO 2 equivalent per year of fugitive methane, outweighing benefits achieved through on-site electricity generation. Systems designed for nutrient removal have the highest greenhouse gas emissions intensity, attributable to energy requirements and N 2 O production, demonstrating current trade-offs between meeting water quality and climate objectives. We analysed key sensitivities and included a geospatial analysis to highlight the scale and distribution of opportunities for reducing life cycle greenhouse gas emissions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

Benchmarking nanoparticulate metal oxide electrocatalysts for the alkaline water oxidation reaction

Nanoparticulate metal-oxide catalysts are among the most prevalent systems for alkaline water oxidation. However, comparisons of the electrochemical performance of these materials have been challenging due to the different methods of attachment, catalyst loadings, and electrochemical test conditions reported in the literature. Here in this paper, we have leveraged a conventional drop-casting method that allows for the successful adhesion of a wide range of nanoparticulate catalysts to glassy-carbon electrode surfaces. We have applied this adhesion method to prepare catalyst films from 16 crystalline metal-oxide nanoparticles with a constant loading of 0.8 mg cm -2 , and evaluated the resulting nanoparticulate films for the oxygen evolution reaction under conditions relevant to an integrated solar fuels device. In general, the activities of the adhered nanoparticulate films are similar to those of thin-film catalysts prepared by electrodeposition or sputtering, achieving 10 mA cm -2 current densities per geometric area at overpotentials of ~0.35–0.5 V.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Benchmarking nitrous oxide adsorption and activation in metal–organic frameworks bearing coordinatively unsaturated metal centers

Anthropogenic emissions of N 2 O, the third most abundant greenhouse gas after CO 2 and CH 4 , are contributing to global climate change. Although metal–organic frameworks (MOFs) have been widely studied as adsorbents for CO 2 and CH 4 , less effort has focused on the use of MOFs to remove N 2 O from emission streams or from air. Further, N 2 O activation would enable its use as an inexpensive oxidant for fine chemical synthesis. Here, in this paper, we identify features that contribute to strong binding and high uptake of N 2 O at coordinatively unsaturated metal sites in the M 2 Cl 2 (btdd) (M = Mn, Co, Ni, Cu; btdd 2– = bis(1,2,3-triazolo[4,5-b],[4',5'-i])dibenzo[1,4]dioxin) and M 2 (dobdc) (M = Mg, Mn, Fe, Co, Ni, Cu, Zn; dobdc 4– = 2,5-dioxido-1,4-benzenedicarboxylate) series of MOFs. Combined experimental and computational studies suggest that N 2 O adsorption at open-metal-sites is primarily based on electrostatic interactions, rather than π-backbonding, causing MOFs with more Lewis acidic metal centers to be superior N 2 O adsorbents. As a result, Mg 2 (dobdc) demonstrates strong binding and record-setting N 2 O uptake (8.75 mmol g –1 at 1 bar and 298 K). Using density functional theory (DFT) to characterize reactive intermediates and transition states, we demonstrate that N 2 O activation to form a M(IV)–oxo species and N 2 is thermodynamically favorable in Mn 2 (dobdc) and Fe 2 (dobdc) but appears to be kinetically limited in Mn 2 (dobdc). Our work lays a foundation for understanding N 2 O adsorption and activation in MOFs, paving the way for the design of promising next-generation materials for N 2 O capture and utilization.

36 MATERIALS SCIENCE↗
DOE OSTI2013

Energy benchmarks for water clusters and ice structures from an embedded many-body expansion

In this paper, we show how an embedded many-body expansion (EMBE) can be used to calculate accurate ab initio energies of water clusters and ice structures using wavefunction-based methods. We use the EMBE described recently by Bygrave et al. [J. Chem. Phys. 137, 164102 (2012)], in which the terms in the expansion are obtained from calculations on monomers, dimers, etc., acted on by an approximate representation of the embedding field due to all other molecules in the system, this field being a sum of Coulomb and exchange-repulsion fields. Our strategy is to separate the total energy of the system into Hartree-Fock and correlation parts, using the EMBE only for the correlation energy, with the Hartree-Fock energy calculated using standard molecular quantum chemistry for clusters and plane-wave methods for crystals. Our tests on a range of different water clusters up to the 16-mer show that for the second-order Møller-Plesset (MP2) method the EMBE truncated at 2-body level reproduces to better than 0.1 mE h /monomer the correlation energy from standard methods. The use of EMBE for computing coupled-cluster energies of clusters is also discussed. For the ice structures Ih, II, and VIII, we find that MP2 energies near the complete basis-set limit reproduce very well the experimental values of the absolute and relative binding energies, but that the use of coupled-cluster methods for many-body correlation (non-additive dispersion) is essential for a full description. Possible future applications of the EMBE approach are suggested.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2013

The Jastrow antisymmetric geminal power in Hilbert space: Theory, benchmarking, and application to a novel transition state

The Jastrow-modified antisymmetric geminal power (JAGP) ansatz in Hilbert space successfully overcomes two key failings of other pairing theories, namely, a lack of inter-pair correlations and a lack of multiple resonance structures, while maintaining a polynomially scaling cost, variational energies, and size consistency. Here, we present efficient quantum Monte Carlo algorithms that evaluate and optimize the JAGP energy for a cost that scales as the fifth power of the system size. We demonstrate the JAGP’s ability to describe both static and dynamic correlation by applying it to bond stretching in H{sub 2}O, C{sub 2}, and N{sub 2} as well as to a novel, multi-reference transition state of ethene. JAGP’s accuracy in these systems outperforms even the most sophisticated single-reference methods and approaches that of exponentially scaling active space methods.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗