Google Inc.

International Business Machines Corporation

Daniel Martin Bikel

Jakob David Uszkoreit

Percy Shuo Liang

Fadi Biadsy

Pedro Jose Moreno Mengibar

Kaisuke Nakajima

Ashish Venugopal

Pravir Kumar Gupta

Ciprian Ioan Chelba

Jeffrey Scott Sorensen

Fernando Pereira

Omer Bar-or

Kapil R Thadini

Maria Shugrina

A language processing system identifies first command input sentences that do not successfully parse by any parsing rule in a set of parsing rules. Each of the parsing rules is associated with an action, and a user device performs the action associated with a parsing rule in response to an input sentence being successfully parsed by the parsing rule. For each of these identified first sentences, the system determines whether the first input sentence has an underserving signal that is indicative of one or more actions being underserved. If the first sentence has the underserving signal, then the first sentence is selected as a candidate input sentence. Each candidate input sentence is provided to an action analysis processes that determines whether a candidate input sentence is to be associated with one action, and upon a positive determination generates a parsing rule for the candidate input sentence.

Identifying underserved command inputs

A method includes accessing command sentences stored in a data store, wherein each command sentence is a collection of n-grams and each command sentence includes at least one n-gram that is a non-terminal n-gram that maps to a non-terminal type, and wherein the command sentences include non-terminal n-grams that collectively map to a plurality of different non-terminal types; for each of the non-terminal types: identifying n-gram spans; determining clusters of the n-gram spans, each cluster including n-gram spans meeting a measure of similarity of n-grams spans that belong to the cluster; and for each cluster of n-gram spans, determining, from the n-gram spans belonging to the cluster, a new non-terminal type to which the terminal n-grams of the n-gram spans map.

Parsing rule generalization by N-gram span clustering

Speech recognition techniques may include: receiving audio; identifying one or more topics associated with audio; identifying language models in a topic space that correspond to the one or more topics, where the language models are identified based on proximity of a representation of the audio to representations of other audio in the topic space; using the language models to generate recognition candidates for the audio, where the recognition candidates have scores associated therewith that are indicative of a likelihood of a recognition candidate matching the audio; and selecting a recognition candidate for the audio based on the scores.

Speech recognition using topic-specific language models

A language processing system identifies sequential command inputs in user session data stored in logs. Each sequence command input is a first command input followed by a second command input. The system determines user actions in response to each command input. For the second command input, an action was taken at the user device in response to the command input, and there is no parsing rule associated with the action that parses to the first command input. If there is a sufficient co-occurrence of the first and second command inputs and the resulting action in the logs, then a parsing rule for the action may be augmented with a rule for the first command input.

Parsing rule augmentation based on query sequence and action co-occurrence

A language processing system uses a lattice parser that semantically parses a command input represented by a lattice. The parser receives a hypotheses space of outputs as encoded in a lattice. Annotations of the input are projected back into the lattice and then lattice parsing is performed to rectify with the annotations. Parsing rules are applied to path fragments in the lattice. The rules that successfully parse from the start node to the end node of the lattice are used to determine whether the command input sentence invokes a specific action, and if so, what arguments are to be passed to the invocation of the action.

Lattice semantic parsing

A language processing system identifies, from log data, command inputs that parsed to a parsing rule associated with an action. If the command input has a signal indicative of user satisfaction, where the signal is derived from data that is not generated from performance of the action (e.g., user interactions with data provided in response to the performance of another, different action; resources identified in response to the performance of another, different action having a high quality score; etc.), then exception data is generated for the parsing rule. The exception data specifies the particular instance of the sentence parsed by the parsing rule, and precludes invocation of the action associated with the rule.

Exceptions to action invocation from parsing rules

A system determines search hypotheses for a search query, each search hypothesis defining a search type and respectively corresponding to a resource corpus of a type that matches the search type; for each search hypothesis, generate a hypothesis search query based on the search query and the search type and submits the hypothesis search query to a search service to determine a search hypothesis score, and for each search hypothesis having a search hypothesis score meeting a search hypothesis threshold, providing search results for the search operation performed for the hypothesis search query determined for the search hypothesis; and for each search hypothesis not having a search hypothesis score meeting a search hypothesis threshold, not providing search results for the search operation performed for the hypothesis search query determined for the search hypothesis.

Corpus specific queries for corpora from search query

A set of benchmark text strings may be classified to provide a set of benchmark classifications. The benchmark text strings in the set may correspond to a benchmark corpus of benchmark utterances in a particular language. A benchmark classification distribution of the set of benchmark classifications may be determined. A respective classification for each text string in a corpus of text strings may also be determined. Text strings from the corpus of text strings may be sampled to form a training corpus of training text strings such that the classifications of the training text strings have a training text string classification distribution that is based on the benchmark classification distribution. The training corpus of training text strings may be used to train an automatic speech recognition (ASR) system.

Sampling training data for an automatic speech recognition system based on a benchmark classification distribution

A language processing system uses annotation services that are external to the language processing system to identify n-grams that identify entities in an input sentence. The n-grams are annotated by the annotation services. The annotations are used to determine which n-grams, if any, correspond to instances of an entity type (e.g., values for a variable or terminals for a non-terminal). After determining which n-grams correspond to entity types, parse initializations are generated for parsing rules and parses for each rule are attempted. The rules that successfully parse are used to determine whether the input sentence invokes a specific action, and if so, what arguments are to be passed to the invocation of the action.

Parsing annotator framework from external services

Parsing rule generalization by n-gram span clustering

Techniques are disclosed for securely classifying or decoding data. By way of example, a method of determining a most likely sequence for a given data set comprises a computer system associated with a first party performing the following steps. An encrypted model is obtained from a second party. The encrypted model is utilized to determine cost values associated with a particular sequence of observed outputs associated with the given data set. The cost values are sent to the second party. At least one index of a minimum cost value determined by the second party from the cost values sent thereto is obtained from the second party. A minimum cost sequence resulting from the at least one index is determined as the most likely sequence.

Growing community of inventors

Mount Kisco, NY, United States of America

Daniel Martin Bikel