Google Inc.

Amazon Technologies, Inc.

General Electric Company

Dirk Padfield

Aaron Michael Donsbach

Fadi Biadsy

Jens Rittscher

Jilin Tu

Gurumurthy Swaminathan

Thomas Lund Dideriksen

Musodiq O Bello

Suren Kumar

Gregory James Nyssen

Wei You

Bryan Huh

Ahmad Yekta

Brian P Jackson

Eric Paul Bennett

Colin Andrew Cherry

Noah Murad

Edward Lo

Victoria Zayats

The technology provides an approach to train translation models that are robust to transcription errors and punctuation errors. The approach includes introducing errors from actual automatic speech recognition and automatic punctuation systems into the source side of the machine translation training data. A method for training a machine translation model includes performing automatic speech recognition on input source audio to generate a system transcript. The method aligns a human transcript of the source audio to the system transcript, including projecting system segmentation onto the human transcript. Then the method performs segment robustness training of a machine translation model according to the aligned human and system transcripts, and performs system robustness training of the machine translation model, e.g., by injecting token errors into training data.

Inverted projection for robust speech translation

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, that facilitate generating stable real-time textual translations in a target language of an input audio data stream that is recorded in a source language. An audio stream that is recorded in a first language is obtained. A partial transcription of the audio can be generated at each time interval in a plurality of successive time intervals. Each partial transcription can be translated into a second language that is different from the first language. Each translated partial transcription can be input to a model that determines whether a portion of an input translated partial transcription is stable. Based on the input translated partial transcription, the model identifies a portion of the translated partial transcription that is predicted to be stable. This stable portion of the translated partial transcription is provided for display on a user device.

Stable real-time translations of audio streams

An automated speech recognition (ASR) transcript of at least a portion of a media content is obtained from an ASR tool. Suggested words are received for corrected words of the ASR transcript of the media content. Features are obtained using at least the suggested words or the corrected words. The features include features relating to sound similarities between the suggested words and the corrected words. The features are input into a machine learning (ML) model to obtain a determination regarding a validity of the suggested words. Responsive to the suggested words constituting a valid suggestion, the suggested words are incorporated into the ASR transcript. At least a portion of the ASR transcript is transmitted to a user device in conjunction with at least a portion of the media content.

False suggestion detection for user-provided content

A method includes receiving a reference audio signal corresponding to reference speech spoken by a target speaker with atypical speech, and generating, by a speaker embedding network configured to receive the reference audio signal as input, a speaker embedding for the target speaker. The speaker embedding conveys speaker characteristics of the target speaker. The method also includes receiving a speech conversion request that includes input audio data corresponding to an utterance spoken by the target speaker associated with the atypical speech. The method also includes biasing, using the speaker embedding generated for the target speaker by the speaker embedding network, a speech conversion model to convert the input audio data corresponding to the utterance spoken by the target speaker associated with atypical speech into an output canonical representation of the utterance spoken by the target speaker.

Speaker embeddings for improved automatic speech recognition

A computing device receives a first audio waveform representing a first utterance and a second utterance. The computing device receives identity data indicating that the first utterance corresponds to a first speaker and the second utterance corresponds to a second speaker. The computing device determines, based on the first utterance, the second utterance, and the identity data, a diarization model configured to distinguish between utterances by the first speaker and utterances by the second speaker. The computing device receives, exclusively of receiving further identity data indicating a source speaker of a third utterance, a second audio waveform representing the third utterance. The computing device determines, by way of the diarization model and independently of the further identity data of the first type, the source speaker of the third utterance. The computing device updates the diarization model based on the third utterance and the determined source speaker.

Adaptive diarization model and user interface

Machine learning techniques are disclosed for training a model to identify each of multiple different classes in images, based on training data where each training image may not be labeled in a complete manner with respect to the classes. The disclosed training techniques use a new label value to indicate when a ground truth value is unknown for a particular class, and do not penalize the machine learning model for output predictions that do not match the label value representing unknown ground truth. The disclosed processes may, for example, be used to train a model to detect each multiple types of image defects based on incomplete information provided by human reviewers who accept and reject images based on whether any of the types of image defects are found.

Model training using incomplete indications of types of defects present in training images

Aspects of the present disclosure relate to machine learning techniques for training a model to identify each of a number of different classes in images, based on training data where each training image may not be labeled in a complete manner with respect to the classes. The disclosed training techniques use a new label value to indicate when a ground truth value is unknown for a particular class, and do not penalize the machine learning network for output predictions that do not match the label value representing unknown ground truth. Some implementations of the training process can be regularized to impose sparsity on predicted classes in order to avoid false positive predictions.

Regularized multi-label classification from partially labeled training data

Systems and methods are described for providing an image validation module. The image validation mobile enables capture, enhancement, validation, and upload of a digital image to a networked computing service, applying criteria that correspond to image validation criteria used by the networked computing service. The image validation mobile may be executed on a mobile computing device, and may authenticate itself to the networked computing service to indicate that digital images have already been validated. The image validation module may provide feedback before, during, or after image capture to enable the capture of valid images, and may provide feedback before, during, or after image enhancement to allow issues that prevent a digital image from passing validation to be addressed.

Image validation

Features are disclosed for processing composite images. Composite images may be received that include a common item such as a t-shirt with different graphics overlaid on the item. Features for detecting such composite images by comparing shape and color features of an uploaded image to previously detected composite images are described. Composite images including the common item may be grouped into clusters. The clustered images can then be processed as a group such as to separate the graphics from the underlying image and to make authorization determinations for inclusion in an online catalog system.

Composite image detection

Features are disclosed for processing composite images. Composite images may be received that include a common item such as a t-shirt with different graphics overlaid on the item. Features for determining the quality of composite images based on processing the image data are provided. Detection of a region that the overlaid graphic covers provides a targeted location for analyzing the underlying image. A quality metric may be determined based on whether, which, and how many features of the item shown in the underlying image are obscured or otherwise modified by the overlaid image.

Composite image quality assurance

Disclosed are various embodiments for improving optical character recognition approaches through the use of gap shifting. A text detection process is performed upon an image to detect a first region of text. A second region that is in line with the first region is shifted to reduce a gap between the first region and the second region, thereby creating a modified image. The text detection process is performed upon the modified image in order to detect text within the second region.

Growing community of inventors

Seattle, WA, United States of America

Dirk Padfield