Soundhound, Inc.

Arteris, Inc.

Soundhound Ai Ip, LLC

Qualcomm Technologies, Inc.

Jonah Probell

Monica Tang

Maisy Wieman

Sudharsan Krishnaswamy

Patricia Pozon Aguayo

Jennifer Hee Young Zhang

Bernard Mont-Reynaud

Pranav Singh

Kheng Khov

Jean-Jacques Lecler

Xavier Van Ruymbeke

Kiran Garaga Lokeswarappa

Philippe Martin

Zili Li

Joe Aung

Alexis Boutillier

Dee Lin

Aliaksei Chapyzhenka

Asif Amirguliyev

A system detects a period of non-voice activity and compares its duration to a cutoff period. The system adapts the cutoff period based on parsing previously-recognized speech of a user that is stored on a user's device or the system, which detects the voice activity, to determine according to a model, such as a machine-learned model, the probability that the speech recognized so far is a prefix to a longer complete utterance. The cutoff period is longer when a parse of previously recognized speech, which is based on the user profile, has a high probability of being a prefix of a longer utterance.

Adapting an utterance cut-off period with user specific profile data

A system and method invoke virtual assistant action, which may comprise an argument. From audio, a probability of an intent is inferred. A probability of a domain and a plurality of variable values may also be inferred. Invoking the action is in response to the intent probability exceeding a threshold. Invoking the action may also be in response to the domain probability exceeding a threshold, a variable value probability exceeding a threshold, detecting an end of utterance, and a specific amount of time having elapsed. The intent probability may increase when the audio includes speech of words with the same meaning in multiple natural languages. Invoking the action may also be conditional on the variable value exceeding its threshold within a certain period of time of the intent probability exceeding its threshold.

Meaning inference from speech audio

A virtual assistant processes natural language expressions according to grammar rules created by domain providers. The virtual assistant uniquely identifies each of a multiplicity of users and stores values of grammar slots filled by natural language expressions from each user. The virtual assistant stores histories of slot values and computes statistics from the history. The virtual assistant provider, or a classification client, provides values of attributes of users as labels for a machine learning classification algorithm. The algorithm processes the grammar slot values and labels to compute probability distributions for unknown attribute values of users. A network effect of users and domain grammars make the virtual assistant useful and provides increasing amounts of data that improve classification accuracy and usefulness.

Classification by natural language grammar slots across domains

A processing system detects a period of non-voice activity and compares its duration to a cutoff period. The system adapts the cutoff period based on parsing previously-recognized speech to determine, according to a model, such as a machine-learned model, the probability that the speech recognized so far is a prefix to a longer complete utterance. The cutoff period is longer when a parse of previously recognized speech has a high probability of being a prefix of a longer utterance.

Adapting an utterance cut-off period based on parse prefix detection

A neural speech-to-meaning system is trained on speech audio expressing specific intents. The system receives speech audio and produces indications of when the speech in the audio matches the intent. Intents may include variables that can have a large range of values, such as the names of places. The neural speech-to-meaning system simultaneously recognizes enumerated values of variables and general intents. Recognized variable values can serve as arguments to API requests made in response to recognized intents. Accordingly, neural speech-to-meaning supports voice virtual assistants that serve users based on API hits.

Neural speech-to-meaning

Training and enhancement of neural network models, such as from private data, are described. A slave device receives a version of a neural network model from a master. The slave accesses a local and/or private data source and uses the data to perform optimization of the neural network model. This can be done such as by computing gradients or performing knowledge distillation to locally train an enhanced second version of the model. The slave sends the gradients or enhanced neural network model to a master. The master may use the gradient or second version of the model to improve a master model.

Neural network training from private data

To train a speech recognizer, such as for recognizing variables in a neural speech-to-meaning system, compute, within an embedding space, a range of vectors of features of natural speech. Generate parameter sets for speech synthesis and synthesis speech according to the parameters. Analyze the synthesized speech to compute vectors in the embedding space. Using a cost function that favors an even spread (minimal clustering) generates a multiplicity of speech synthesis parameter sets. Using the multiplicity of parameter sets, generate a multiplicity of speech of known words that can be used as training data for speech recognition.

Synthesizing speech recognition training data

A method of predicting a person's interests is provided. The method includes receiving geolocation information about a user location, reading, from a database of interpretations, at least one interpretation of an expression made in close proximity to the location, reading, from a database of ad bids, a plurality of ad bids comprising interpretations, comparing the interpretation from the database to the interpretations of the ad bids to select a most valuable ad bid having an interpretation that matches the interpretation of an expression made in close proximity to the location, and presenting an ad associated with the most valuable ad bid, wherein the interpretation is from a natural language expression.

Geographical mapping of interpretations of natural language expressions

Simulation or calculation to estimate activity per unit in a chip design, combined with estimation of the specific location or region in which the unit logic will be finally placed, provides for calculation of an estimation of the activity distribution within the floorplan. Activity distribution estimation can be performed with fine granularity (at a gate level), at coarse granularity (at a macro level), or at an intermediate granularity (at a network-on-chip unit level). The estimation is displayed, visually, to a user of a design tool. Furthermore, the estimation is used to make manual or automatic optimizations of the floorplan and the location and configuration of units within the floorplan.

System and method for estimation of chip floorplan activity

A speech-based human-machine interface that parses words spoken to detect a complete parse and, responsive to so detecting, computes a hypothesis as to whether the words are a prefix to another complete parse. The duration of no voice activity period to determine an end of a sentence depends on the prefix hypothesis. The user's typical speech speed profile and a short-term measure of speech speed also scale the period. Speech speed is measured by the time between words, and the period scaling uses a continuously adaptive algorithm. The system uses a longer cut-off period after a system wake-up event but before it detects any voice activity.

Parse prefix-detection in a human-machine interface

Estimation of chip floorplan activity distribution

A Network-on-Chip (NoC) link with an upstream bypassable narrowing serialization adapter and a downstream bypassable widening serialization adapter. The serialization adapters are normally bypassed. To avoid sending information on broken wires, bypassing is disabled so that information is serialized to only a portion of the link. Serialization can be applied to any portion of a link down to as little as one bit wire.

Dynamic link serialization in network-on-chip

Speech synthesis chooses pronunciations of words with multiple acceptable pronunciations based on an indication of a personal, class-based, or global preference or an intended non-preferred pronunciation. A speaker's words can be parroted back on personal devices using preferred pronunciations for accent training. Degrees of pronunciation error are computed and indicated to the user in a visual transcription or audibly as word emphasis in parroted speech. Systems can use sets of phonemes extended beyond those generally recognized for a language. Speakers are classified in order to choose specific phonetic dictionaries or adapt global ones. User profiles maintain lists of which pronunciations are preferred among ones acceptable for words with multiple recognized pronunciations. Systems use multiple correlations of word preferences across users to predict use preferences of unlisted words. Speaker-preferred pronunciations are used to weight the scores of transcription hypotheses based on phoneme sequence hypotheses in speech engines.

Growing community of inventors

Alviso, CA, United States of America

Jonah Probell