Descriptive topic modeling with lda on bags of utterance clusters

Abstract

The present disclosure relates to a system, a method, and a product for intent discovery. The system includes a processor in communication with a memory storing instructions. When the processor executes the instructions, the instructions are configured to cause the processor to: obtain documents comprising a set of utterances, extract the set of utterances from the documents, generate a set of utterance embeddings based on the set of utterances, clusterize the set of utterance embeddings to obtain a plurality of clusters, obtain a cluster label for each cluster, encode each document based on a number of times each utterance cluster identifier (ID) appears to obtain an encoded document, perform latent Dirichlet allocation (LDA) on the encoded documents to obtain K topics, and each topic corresponding to a list of key clusters with cluster IDs, and for each topic, replace the cluster IDs with the cluster labels.