Second International Workshop on Symbolic-Neural Learning (SNL-2018)
July 5-6, 2018
Nagoya Congress Center (Nagoya, Japan)
Semantic Graph Embeddings and a Neural Language Model for Word Sense Disambiguation
Marc Evrard (Toyota Technological Institute), Makoto Miwa (Toyota Technological Institute), and Yutaka Sasaki (Toyota Technological Institute)
Abstract:
Word sense disambiguation (WSD) consists in computationally identifying the meaning of words in the context in which they occur. WSD is a long-standing natural language processing (NLP) problem and is at the core of language understanding. We focus on “all words WSD” tasks in this work. It requires determining the linking between all content words (word presenting a substantive meaning—as opposed to functional words) in a document and their corresponding sense stored in a semantic graph, respectively. The state of the art for the task of all words WSD is achieved by resorting to graph-based techniques applied on large semantic graphs in order to alleviate their typical sparsity problems. The linking between the words in the document and their sense in the semantic graph is also performed by graph-based techniques that assert a high-coherence semantic interpretation of the document. These methods allow surpassing the strong most-frequent-sense baseline. Their high-coherence interpretation allows them to well capture the global context of the document, but fail to capture more fine-grained senses that depend on the local context at the sentence level. This work investigates a way to improve this limitation by resorting to continuous vector representations of the documents and of the semantic graph, in order to allow for both the global and local modeling of the document’s context, as well as a more fine-grained sense discrimination. A neural language model is used to embed the document, and a new method to perform the semantic graph embeddings is proposed and compared to other established techniques. In order to avoid the complex mapping between the sentence embeddings' and the semantic graph embeddings' spaces, a sampling procedure is applied. The quality of the graph embedding techniques is evaluated on intrinsic evaluations, such as word similarity, as well as on the disambiguation process.