Google Inc.

Izhak Shafran

Tara N Sainath

Michiel A U Bacchiani

Samuel Bengio

Ehsan Variani

Mirkó Visontai

Christopher Walter George Thornton

Arun Narayanan

Russell John Wyatt Skerry-Ryan

Richard Cameron Rose

Thomas Edward Bagby

Joseph Caroselli

Andrew William Senior

Hasim Sak

Hagen Soltau

Nan Du

Laurent El Shafey

Yu-hui Chen

Linh Mai Tran

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for extracting entities from conversation transcript data. One of the methods includes obtaining a conversation transcript sequence, processing the conversation transcript sequence using a span detection neural network configured to generate a set of text token spans; and for each text token span: processing a span representation using an entity name neural network to generate an entity name probability distribution over a set of entity names, each probability in the entity name probability distribution representing a likelihood that a corresponding entity name is a name of the entity referenced by the text token span; and processing the span representation using an entity status neural network to generate an entity status probability distribution over a set of entity statuses.

Learning to extract entities from conversations with neural networks

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing audio data using neural networks.

Joint automatic speech recognition and speaker diarization

Utilizing an adaptive multichannel technique to mitigate reverberation present in received audio signals, prior to providing corresponding audio data to one or more additional component(s), such as automatic speech recognition (ASR) components. Implementations disclosed herein are 'adaptive', in that they utilize a filter, in the reverberation mitigation, that is online, causal and varies depending on characteristics of the input. Implementations disclosed herein are 'multichannel', in that a corresponding audio signal is received from each of multiple audio transducers (also referred to herein as &#x201c;microphones&#x201d;) of a client device, and the multiple audio signals (e.g., frequency domain representations thereof) are utilized in updating of the filter&#x2014;and dereverberation occurs for audio data corresponding to each of the audio signals (e.g., frequency domain representations thereof) prior to the audio data being provided to ASR component(s) and/or other component(s).

Adaptive multichannel dereverberation for automatic speech recognition

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for speech recognition using complex linear projection are disclosed. In one aspect, a method includes the actions of receiving audio data corresponding to an utterance. The method further includes generating frequency domain data using the audio data. The method further includes processing the frequency domain data using complex linear projection. The method further includes providing the processed frequency domain data to a neural network trained as an acoustic model. The method further includes generating a transcription for the utterance that is determined based at least on output that the neural network provides in response to receiving the processed frequency domain data.

Complex linear projection for acoustic modeling

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for speech recognition using complex evolution recurrent neural networks. In some implementations, audio data indicating acoustic characteristics of an utterance is received. A first vector sequence comprising audio features determined from the audio data is generated. A second vector sequence is generated, as output of a first recurrent neural network in response to receiving the first vector sequence as input, where the first recurrent neural network has a transition matrix that implements a cascade of linear operators comprising (i) first linear operators that are complex-valued and unitary, and (ii) one or more second linear operators that are non-unitary. An output vector sequence of a second recurrent neural network is generated. A transcription for the utterance is generated based on the output vector sequence generated by the second recurrent neural network. The transcription for the utterance is provided.

Complex evolution recurrent neural networks

Linear transformation for speech recognition modeling

Methods, systems, and apparatus, including computer programs encoded on computer storage media for modeling phonemes. One method includes receiving an acoustic sequence, the acoustic sequence representing an utterance, and the acoustic sequence comprising a respective acoustic feature representation at each of a plurality of time steps; for each of the plurality of time steps: processing the acoustic feature representation through each of one or more recurrent neural network layers to generate a recurrent output; processing the recurrent output using a softmax output layer to generate a set of scores, the set of scores comprising a respective score for each of a plurality of context dependent vocabulary phonemes, the score for each context dependent vocabulary phoneme representing a likelihood that the context dependent vocabulary phoneme represents the utterance at the time step; and determining, from the scores for the plurality of time steps, a context dependent phoneme representation of the sequence.

Context-dependent modeling of phonemes

Growing community of inventors

Menlo Park, CA, United States of America

Izhak Shafran