Soundhound, Inc.

Soundhound Ai Ip, LLC

Pranav Singh

Keyvan Mohajer

Bernard Mont-Reynaud

Kamyar Mohajer

Olivia Bettaglio

Yilun Zhang

Jonah Probell

Kheng Khov

Eunjee Na

Bernard Mont-reynaud

Saraswati Mishra

Mohammadreza Fazeli

Luke Lefebure

Aspects include methods, systems, and computer-program products providing virtual assistant domain functionality. A natural language query including one or more words is received. A collection of natural language modules is accessed. The collection natural language modules are configured to process sets of natural language queries. A natural language module, from the collection of natural language modules, is identified to interpret the natural language query. An interpretation of the natural language query is computed using the identified natural language module. A response to the natural language query is returned using the computed interpretation.

Virtual assistant domain functionality

Automatically generating sentences that a user can say to invoke a set of defined actions performed by a virtual assistant are disclosed. A sentence is received and keywords are extracted from the sentence. Based on the keywords, additional sentences are generated. A classifier model is applied to the generated sentences to determine a sentence that satisfies a threshold. In the situation a sentence satisfies the threshold, an intent associated with the classifier model can be invoked. In the situation the sentences fail to satisfy the classifier model, the virtual assistant can attempt to interpret the received sentence according to the most likely intent by invoking a sentence generation model fine-tuned for a particular domain, generate additional sentences with a high probability of having the same intent and fulfill the specific action defined by the intent.

Domain specific neural sentence generator for multi-domain virtual assistants

Methods and systems for automatically generating sample phrases or sentences that a user can say to invoke a set of defined actions performed by a virtual assistant are disclosed. By enabling finetuned general-purpose natural language models, the system can generate potential and accurate utterance sentences based on extracted keywords or the input utterance sentence. Furthermore, domain-specific datasets can be used to train the pre-trained, general-purpose natural language models via unsupervised learning. These generated sentences can improve the efficiency of configuring a virtual assistant. The system can further optimize the effectiveness of a virtual assistant in understanding the user, which can enhance the user experience of communicating with it.

Neural sentence generator for virtual assistants

Methods and systems for correction of a likely erroneous word in a speech transcription are disclosed. By evaluating token confidence scores of individual words or phrases, the automatic speech recognition system can replace a low-confidence score word with a substitute word or phrase. Among various approaches, neural network models can be used to generate individual confidence scores. Such word substitution can enable the speech recognition system to automatically detect and correct likely errors in transcription. Furthermore, the system can indicate the token confidence scores on a graphic user interface for labeling and dictionary enhancement.

Token confidence scores for automatic speech recognition

Systems and methods are provided for natural language processing using neural network models and natural language virtual assistants. The system and method include receiving a natural language phrase including a word sequence, computing corresponding error probabilities that the words are errors, and for a word with a corresponding error probability above a threshold, then computing a replacement phrase with a low error probability to provide a response from the virtual assistant depending on the replacement phrase.

System and method for correction of a query using a replacement phrase

A machine learning system for a digital assistant is described, together with a method of training such a system. The machine learning system is based on an encoder-decoder sequence-to-sequence neural network architecture trained to map input sequence data to output sequence data, where the input sequence data relates to an initial query and the output sequence data represents canonical data representation for the query. The method of training involves generating a training dataset for the machine learning system. The method involves clustering vector representations of the query data samples to generate canonical-query original-query pairs in training the machine learning system.

Machine learning system for digital assistants

Systems and methods are provided for systematically finding and fixing automatic speech recognition (ASR) mistranscriptions and natural language understanding (NLU) misinterpretations and labeling data for machine learning. High similarity of non-identical consecutive queries indicates ASR mistranscriptions. Consecutive queries with close vectors in a semantic embedding space indicates NLU misinterpretations. Key phrases and barge-in also indicate errors. Only queries within a short amount of time are considered.

System and method for detection and correction of a query

A method of predicting a person's interests is provided. The method includes receiving geolocation information about a user location, reading, from a database of interpretations, at least one interpretation of an expression made in close proximity to the location, reading, from a database of ad bids, a plurality of ad bids comprising interpretations, comparing the interpretation from the database to the interpretations of the ad bids to select a most valuable ad bid having an interpretation that matches the interpretation of an expression made in close proximity to the location, and presenting an ad associated with the most valuable ad bid, wherein the interpretation is from a natural language expression.

Geographical mapping of interpretations of natural language expressions

A platform provides for developers of applications, such as devices, with natural language interfaces to configure the availability of vertical domain modules in applications. Modules can include grammars for parsing natural language expressions and interfaces to data sources. Third party developers can create modules with pricing models for their usage or access to their data. Device developers can browse or search available modules and test their performance for specific queries. The platform provides for devices users to access the chosen modules as configured by device developers and for charging and payment between users, application developers, and module developers.

Modular virtual assistant platform

A speech recognition and natural language understanding system performs insertion, deletion, and replacement edits of tokens at positions with low probabilities according to both a forward and a backward statistical language model (SLM) to produce rewritten token sequences. Multiple rewrites can be produced with scores depending on the probabilities of tokens according to the SLMs. The rewritten token sequences can be parsed according to natural language grammars to produce further weighted scores. Token sequences can be rewritten iteratively using a graph-based search algorithm to find the best rewrite. Mappings of input token sequences to rewritten token sequences can be stored in a cache, and searching for a best rewrite can be bypassed by using cached rewrites when present. Analysis of various initial token sequences that produce the same new rewritten token sequence can be useful to improve natural language grammars.

Bidirectional probabilistic natural language rewriting and selection

An accurate thought map is created by recording people's many utterances of natural language expressions together with the location at which each expression was made. The expressions are input into a Natural Language Understanding system including a semantic parser, and the resulting interpretations stored in a database with the geolocation of the speaker. Emotions, concepts, time, user identification, and other interesting information may also be detected and stored. Interpretations of related expressions may be linked in the database. The database may be indexed and filtered according to multiple aspects of interpretations such as geolocation ranges, time ranges or other criteria, and analyzed according to multiple algorithms. The analyzed results may be used to render map displays, determine effective locations for advertisements, preemptively fetch information for users of mobile devices, and predict the behavior of individuals and groups of people.

Predicting human behavior by machine learning of natural language interpretations

The present invention extends to methods, systems, and computer program products for a natural language module store. In general, the invention can be used to manage natural language modules offered through a natural language module store. Natural language module (NLM) developers can post NLMs at a NLM store to make the NLMs available for use by others. Developers can select NLMs for inclusion in natural language interpreters (NLIs) containing (and possibly integrating the functionality of) one or more NLMs. Prior to selecting a NLM, a developer can search or browse NLMs to identify an appropriate NLM. Optionally, a developer can test a NLM in the NLM store prior to inclusion in an NLI. For example, multiple NLMs purporting to provide the same specified natural language functionality can be tested relative to one another prior to selection of one of the NLMs for inclusion in an NLI.

Growing community of inventors

Sunnyvale, CA, United States of America

Pranav Singh