Hierarchical multi-task term embedding learning for synonym prediction

Abstract

Due to the high language use variability in real-life, manual construction of semantic resources to cover all synonyms is prohibitively expensive and may result in limited coverage. Described herein are systems and methods that automate the process of synonymy resource development, including both formal entities and noisy descriptions from end-users. Embodiments of a multi-task model with hierarchical task relationship are presented that learn more representative entity/term embeddings and apply them to synonym prediction. In model embodiments, a skip-gram word embedding model is extended by introducing an auxiliary task 'neighboring word/term semantic type prediction' and hierarchically organize them based on the task complexity. In one or more embodiments, existing term-term synonymous knowledge is integrated into the word embedding learning framework. Embeddings trained from the multi-task model embodiments yield significant improvement for entity semantic relatedness evaluation, neighboring word/term semantic type prediction, and synonym prediction compared with baselines.