Search NASA⌕ Search

DOE OSTI · 3013845

Explaining word embeddings with perfect fidelity: a case study in predicting research impact

Abstract

The best-performing approaches for scholarly document quality prediction are based on embedding models. In addition to their performance when used in classifiers, embedding models can also provide predictions even for words that were not contained in the labelled training data for the classification model, which is important in the context of the ever-evolving research terminology. Although model-agnostic explanation methods, such as Local interpretable model-agnostic explanations, can be applied to explain machine learning classifiers trained on embedding models, these produce results with questionable correspondence to the model. We introduce a new feature importance method, Self-Model Entities Rated (SMER), for logistic regression-based classification models trained on word embeddings. We show that SMER has theoretically perfect fidelity with the explained model, as the average of logits of SMER scores for individual words (SMER explanation) exactly corresponds to the logit of the prediction of the explained model. Quantitative and qualitative evaluation is performed through five diverse experiments conducted on 50,000 research articles (papers) from the CORD-19 corpus. In conclusion, through an AOPC curve analysis, we experimentally demonstrate that SMER produces better explanations than LIME, SHAP and global tree surrogates.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Dvořáčková, Lucie [Prague University of Economics and Business, Prague (Czech Republic)], Joachimiak, Marcin P. [Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)], Černý, Michal [Prague University of Economics and Business, Prague (Czech Republic)], Kubecová, Adriana [Prague University of Economics and Business, Prague (Czech Republic)], Sklenák, Vilém [Prague University of Economics and Business, Prague (Czech Republic)], Kliegr, Tomáš [Prague University of Economics and Business, Prague (Czech Republic)]. 2025-10-30. Explaining word embeddings with perfect fidelity: a case study in predicting research impact. https://doi.org/10.1007/s10994-025-06870-6

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

Anomalous softening of 3D printed elastomeric foam irradiated under compressive strain

Elastomeric foam is an essential component in many industrial and technological settings, primarily as thermal insulators and as positional/mechanical support cushions. In particular, silicone foam is utilized in harsh environments due to exceptional thermal and chemical stability. Under service conditions within certain applications such material gets exposed to a high dosage of gamma radiation, which can permanently alter the material’s structural and mechanical response properties. Most studies on gamma-exposure under inert or oxidative atmosphere indicate hardening of silicone foam, which is attributed to radiation-induced enhancement in chemical cross-linking. Here we report two contrasting effects depending on whether (non-oxidative) radiation exposure is carried out with the foam under zero or finite compressive strain. While in the former case we observe radiation-hardening consistent with previous studies, in the latter case (50% porous foam under 30% uniaxial compression) we see a monotonic decrease in Young’s modulus with increasing dosage, although solvent swelling experiments on the constituent rubber indicate a net increase in cross-link density independent of the state of strain. We quantitatively model all dose-dependent data using the Ogden Hyperfoam strain-energy function within the framework of Tobolsky two-network scheme and attribute the above anomaly to a combined effect of radiation-induced thickness change (compression set) and inherent nonlinearity in the foam’s stress-strain response.

Coarse-grained models↗