Automated discourse phrase discovery for generating an improved language model of a digital assistant

Abstract

Disclosed are techniques for the automated discovery and extraction of discourse phrases, or in other words terms that representative of a topic or concept communicated via a plurality of electronic documents, to facilitate the generation of a language model that is applicable to interpreting commands for invoking application-based actions via a digital assistant device. An electronic document corpus can be generated having a plurality of electronic documents that are clustered based on entropy, among other things. The clusters can be associated with a corresponding plurality of cluster attractors that are generally representative of a context of the documents included therein. The clusters can be grouped based on determined relationships, such that the groups can be further analyzed and terms or phrases representative of a discourse associated with the grouped documents can be determined. The various groups and clusters of documents can be saved and indexed, among other things, to facilitate more efficient searching of the electronic document corpus to interpret received inputs, such as commands received via a digital assistant device.