System and method to generate interpretable embeddings for domain specific small corpus

Abstract

A method and systems for generating interpretable and embeddings for a domain-specific small corpus of text-based documents are described. A processing module may obtain the plurality of text-based documents and perform a basic cleaning of each of the plurality of text-based documents. Further, the semantic infusion module may generate the semantically infused corpus using the semantic infusion technique. An embedding generation module is configured to compute the optimal dimensionality for the infused corpus and generate the infused optimal dimensional embeddings using word2vec technique. Further, the embedding generation module is configured to generate baseline optimal dimensional embeddings which can be used to evaluate in terms of interpretability and downstream classification task performance.