Embedding Calculus with Nonword Properties Improves Word Sense Disambiguation

Abstract

The present study concerns word sense disambiguation in neural language models using the diagnostic classifiers and hierarchical lexical network. First, we conducted an experiment to see whether the neural models are capable of detecting ambiguous nouns and how they do so. Secondly, we carried out an experiment to verify whether the neural models can identify a specific sense of a lexeme and how they do so. For these experiments, we made use of Word2Vec and FastText as the fixed embedding models and BERT as the contextualized model. In addition, we examined the uniformed and weighted sum method by adding nonword properties (senses). In the case of ambiguity detection, BERT with the general embedding showed better performance than the other models. In regards to sense class detection, BERT with nonword properties showed the best performance on lexemes with numerous senses.