Google Inc.

Kevin I Kilgour

Matthew Nirvan Sharifi

Beat Gfeller

Dominik Roblek

Gabriel Taubman

Blaise Hilary Aguera-Arcas

Brandon Charles Barbello

Julian James Odell

Thomas Weedon Hume

Christopher Thornton

Jan Heinrich Althaus

Mihajlo Velimirović

James David Lyon

Katsiaryna Naliuka

Marvin Karl Ritter

Victor Carbune

Michael Golikov

Marco Tagliasacchi

Aren Jansen

Nicolo D'Ercole

Justin Lu

Scott Wisdom

Ondrej Skopek

Daniel Valcarce

Mohamad Hassan Rom

Qingqing Huang

Example methods include receiving training data comprising a plurality of audio clips and a plurality of textual descriptions of audio. The methods include generating a shared representation comprising a joint embedding. An audio embedding of a given audio clip is within a threshold distance of a text embedding of a textual description of the given audio clip. The methods include generating, based on the joint embedding, a conditioning vector and training, based on the conditioning vector, a neural network to: receive (i) an input audio waveform, and (ii) an input comprising one or more of an input textual description of a target audio source in the input audio waveform, or an audio sample of the target audio source, separate audio corresponding to the target audio source from the input audio waveform, and output the separated audio corresponding to the target audio source in response to the receiving of the input.

Conditioned separation of arbitrary sounds based on machine learning models

Some implementations process, using warm word model(s), a stream of audio data to determine a portion of the audio data that corresponds to particular word(s) and/or phrase(s) (e.g., a warm word) associated with an assistant command, process, using an automatic speech recognition (ASR) model, a preamble portion of the audio data (e.g., that precedes the warm word) and/or a postamble portion of the audio data (e.g., that follows the warm word) to generate ASR output, and determine, based on processing the ASR output, whether a user intended the assistant command to be performed. Additional or alternative implementations can process the stream of audio data using a speaker identification (SID) model to determine whether the audio data is sufficient to identify the user that provided a spoken utterance captured in the stream of audio data, and determine if that user is authorized to cause performance of the assistant command.

Contextual suppression of assistant command(s)

In general, the subject matter described in this disclosure can be embodied in methods, systems, and program products. A computing device stores reference song characterization data and receives digital audio data. The computing device determines whether the digital audio data represents music and then performs a different process to recognize that the digital audio data represents a particular reference song. The computing device then outputs an indication of the particular reference song.

Determining that audio includes music and then identifying the music as a particular song

In general, the subject matter described in this disclosure can be embodied in methods, systems, and program products for indicating a reference song. A computing device stores reference song characterization data that identifies a plurality of audio characteristics for each reference song in a plurality of reference songs. The computing device receives digital audio data that represents audio recorded by a microphone, converts the digital audio data from time-domain format into frequency-domain format, and uses the digital audio data in the frequency-domain format in a music-characterization process. In response to determining that characterization values for the digital audio data are most relevant to characterization values for a particular reference song, the computing device outputs an indication of the particular reference song.

Identifying music as a particular song

Growing community of inventors

Zurich, Switzerland

Kevin I Kilgour