DOE OSTI · 3013845
Explaining word embeddings with perfect fidelity: a case study in predicting research impact
Abstract
The best-performing approaches for scholarly document quality prediction are based on embedding models. In addition to their performance when used in classifiers, embedding models can also provide predictions even for words that were not contained in the labelled training data for the classification model, which is important in the context of the ever-evolving research terminology. Although model-agnostic explanation methods, such as Local interpretable model-agnostic explanations, can be applied to explain machine learning classifiers trained on embedding models, these produce results with questionable correspondence to the model. We introduce a new feature importance method, Self-Model Entities Rated (SMER), for logistic regression-based classification models trained on word embeddings. We show that SMER has theoretically perfect fidelity with the explained model, as the average of logits of SMER scores for individual words (SMER explanation) exactly corresponds to the logit of the prediction of the explained model. Quantitative and qualitative evaluation is performed through five diverse experiments conducted on 50,000 research articles (papers) from the CORD-19 corpus. In conclusion, through an AOPC curve analysis, we experimentally demonstrate that SMER produces better explanations than LIME, SHAP and global tree surrogates.
Explore related subjects
Keep this discovery
Explore connections, maps & timelines
Dvořáčková, Lucie [Prague University of Economics and Business, Prague (Czech Republic)], Joachimiak, Marcin P. [Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States)], Černý, Michal [Prague University of Economics and Business, Prague (Czech Republic)], Kubecová, Adriana [Prague University of Economics and Business, Prague (Czech Republic)], Sklenák, Vilém [Prague University of Economics and Business, Prague (Czech Republic)], Kliegr, Tomáš [Prague University of Economics and Business, Prague (Czech Republic)]. 2025-10-30. Explaining word embeddings with perfect fidelity: a case study in predicting research impact. https://doi.org/10.1007/s10994-025-06870-6
Cite the original work for its findings. Save a collection to share your selection of sources.