Google Inc.

Waymo LLC

Other

Dominik Roblek

Matthew Nirvan Sharifi

Annie Chen

Gheorghe Postelnicu

Yaniv Bernstein

George Tzanetakis

Julian James Odell

Marco Tagliasacchi

Matthew Wiseman

Beat Gfeller

David Petrou

Yossi Matias

Thomas Bugnon

Jorge Pereira

Cong Li

Mihajlo Velimirović

Gabriel Taubman

Blaise Hilary Aguera-Arcas

Brandon Charles Barbello

Thomas Weedon Hume

Christopher Thornton

Neil Zeghidour

Jan Heinrich Althaus

Kevin I Kilgour

Félix De Chaumont Quitry

James David Lyon

Katsiaryna Naliuka

Marvin Karl Ritter

Jay Yagnik

Sergey Ioffe

Ioannis Agiomyrgiannakis

Daniel Jules Switkin

James Lin

Kevin Kilgour

Vera Dron

Owen Daniel Otto

Raziel Alvarez Guevara

Christian Szegedy

Lars Fabian Krüger

Ben M Shahshahani

Zalán Borsos

Karolis Misiunas

Matthew Sharifi

Sandeep Siddhartha

Hannah Muckenhirn

Christian Frank

Jacek Slawosz Jurewicz

Paul Joyce

Yunpeng Li

Tim C Jones

Ehud Graff

Oleg Rybakov

Victor Ungureanu

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for audio processing using neural networks. One of the systems includes multiple neural network layers, wherein the neural network system is configured to receive time domain features of an audio sample and to process the time domain features to generate a neural network output for the audio sample, the plurality of neural network layers comprising: a frequency-transform (F-T) layer that is configured to apply a transformation defined by a set of F-T layer parameters that transforms a window of time domain features into frequency domain features; and one or more other neural network layers having respective layer parameters, wherein the one or more neural network layers are configured to process frequency domain features to generate a neural network output.

Audio processing with neural networks

Methods, systems and apparatus, including computer programs encoded on computer storage media. According to one aspect, there is provided a method comprising: receiving a new input; processing the new input using an encoder neural network to generate a feature vector representing the new input; and generating a coded representation of the feature vector using a sequence of vector quantizers that are each associated with a respective codebook of code vectors, wherein the coded representation of the feature vector identifies a plurality of code vectors, including a respective code vector from the codebook of each vector quantizer, that define a quantized representation of the feature vector.

Generating coded data representations using neural networks and vector quantizers

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing an input audio waveform using a generator neural network to generate an output audio waveform. In one aspect, a method comprises: receiving an input audio waveform; processing the input audio waveform using an encoder neural network to generate a set of feature vectors representing the input audio waveform; and processing the set of feature vectors representing the input audio waveform using a decoder neural network to generate an output audio waveform that comprises a respective output audio sample for each of a plurality of output time steps.

Generating audio waveforms using encoder and decoder neural networks

Systems and methods for training a machine-learned model are provided. A method can include can include obtaining an unlabeled audio signal, sampling the unlabeled audio signal to select one or more sampled slices, inputting the one or more sampled slices into a machine-learned model, receiving, as an output of the machine-learned model, one or more determined characteristics associated with the audio signal, determining a loss function for the machine-learned model based at least in part on a difference between the one or more determined characteristics and one or more corresponding ground truth characteristics of the audio signal, and training the machine-learned model from end to end based at least in part on the loss function. The one or more determined characteristics can include one or more reconstructed portions of the audio signal temporally adjacent to the one or more sampled slices or an estimated distance between two sampled slices.

Self-supervised audio representation learning for mobile devices

A method of training a custom hotword model includes receiving a first set of training audio samples. The method also includes generating, using a speech embedding model configured to receive the first set of training audio samples as input, a corresponding hotword embedding representative of a custom hotword for each training audio sample of the first set of training audio samples. The speech embedding model is pre-trained on a different set of training audio samples with a greater number of training audio samples than the first set of training audio samples. The method further includes training the custom hotword model to detect a presence of the custom hotword in audio data. The custom hotword model is configured to receive, as input, each corresponding hotword embedding and to classify, as output, each corresponding hotword embedding as corresponding to the custom hotword.

Training keyword spotters

Methods, systems, and apparatus, for generated labeled training examples for machine learning. In one aspect, a method includes receiving sets of audio recordings by a user device. For each set of audio recordings, each audio recording in the set is recorded over a respective separate microphone in the user device during a particular time interval, and each particular time interval is different for each set of audio recordings. For each set of audio recordings, a detector determines whether an audio recording in the set of audio recordings includes a particular audio feature, and whether another one of the audio recordings does not include the particular audio feature. For each set of audio recordings determined to include an audio recording that includes the particular audio feature and to include another audio recording that does not include the particular audio feature, a labeled training example is generated.

Automated mining of real-world audio training data

Systems and methods are provided for a personalized entity repository. For example, a computing device comprises a personalized entity repository having fixed sets of entities from an entity repository stored at a server, a processor, and memory storing instructions that cause the computing device to identify fixed sets of entities that are relevant to a user based on context associated with the computing device, rank the fixed sets by relevancy, and update the personalized entity repository using selected sets determined based on the rank and on set usage parameters applicable to the user. In another example, a method includes generating fixed sets of entities from an entity repository, including location-based sets and topic-based sets, and providing a subset of the fixed sets to a client, the client requesting the subset based on the client's location and on items identified in content generated for display on the client.

Personalized entity repository

Systems and methods for media aggregation are disclosed herein. The system includes a media system that can transform media items into one aggregated media item. A synchronization component synchronizes media items with respect to time. The synchronized media items can be analyzed and transformed into an aggregated media item for storage and/or display. In one implementation, the aggregated media item is capable of being displayed in multiple ways to create an enhanced and customizable viewing and/or listening experience.

Aggregation of related media content

Methods, systems and apparatus, including computer programs encoded on computer storage media. One of the methods includes receiving an audio waveform that includes a respective audio sample for each of a plurality of time steps, processing the audio waveform using an encoder neural network to generate a plurality of feature vectors representing the audio waveform, generating a respective coded representation of each of the plurality of feature vectors using a plurality of vector quantizers that are each associated with a respective codebook of code vectors, wherein the respective coded representation of each feature vector identifies a plurality of code vectors, including a respective code vector from the codebook of each vector quantizer, that define a quantized representation of the feature vector, and generating a compressed representation of the audio waveform by compressing the respective coded representation of each of the plurality of feature vectors.

Compressing audio waveforms using neural networks and vector quantizers

Example embodiments relate to techniques for training artificial neural networks or oilier machine-learning encoders to accurately predict the pitch of input audio samples in a semitone or otherwise logarithmically-scaled pitch space. An example method may include generating, from a sample of audio data, two training samples by applying two different pitch shifts to the sample of audio training data. This can be done by converting the sample of audio data into the frequency domain and then shifting the transformed data. These known shifts are then compared to the predicted pitches generated by applying the two training samples to the encoder. The encoder is then updated based on the comparison, such that the relative pitch output by the encoder is improved with respect to accuracy. One or more audio samples, labeled with absolute pitch values, can then be used to calibrate the relative pitch values generated by the trained encoder.

Self-supervised pitch estimation

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for verifying an identity of a user. The methods, systems, and apparatus include actions of receiving a request for a verification phrase for verifying an identity of a user. Additional actions include, in response to receiving the request for the verification phrase for verifying the identity of the user, identifying subwords to be included in the verification phrase and in response to identifying the subwords to be included in the verification phrase, obtaining a candidate phrase that includes at least some of the identified subwords as the verification phrase. Further actions include providing the verification phrase as a response to the request for the verification phrase for verifying the identity of the user.

Segment-based speaker verification using dynamically generated phrases

Growing community of inventors

Meilen, Switzerland

Dominik Roblek