Google Inc.

Gm Global Technolgoy Operations, Inc.

Joseph Edward Roth

Sourish Chaudhuri

Sharadh Ramaswamy

Xiaoming Liu

Michael J Bliss

Yousef Atoum

Yunfei Zhang

Ondrej Klejch

Implementations described herein relate to methods, devices, and computer-readable media to perform gating for video analysis. In some implementations, a computer-implemented method includes obtaining a video comprising a plurality of frames and corresponding audio. The method further includes performing sampling to select a subset of the plurality of frames based on a target frame rate and extracting a respective audio spectrogram for each frame in the subset of the plurality of frames. The method further includes reducing resolution of the subset of the plurality of frames. The method further includes applying a machine-learning based gating model to the subset of the plurality of frames and corresponding audio spectrograms and obtaining, as output of the gating model, an indication of whether to analyze the video to add one or more video annotations.

Gating model for video analysis

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating predictions for whether a target person is speaking during a portion of a video. In one aspect, a method includes obtaining one or more images which each depict a mouth of a given person at a respective time point. The images are processed using an image embedding neural network to generate a latent representation of the images. Audio data corresponding to the images is processed using an audio embedding neural network to generate a latent representation of the audio data. The latent representation of the images and the latent representation of the audio data is processed using a recurrent neural network to generate a prediction for whether the given person is speaking.

Speaking classification using audio-visual data

A method and apparatus for providing trailer information are provided. The method includes detecting a p coupler of the trailer in the image of the rear-facing camera; detecting a position of the coupler in the received image; and determining a distance between the detected position of the coupler of the trailer and a hitch of vehicle. The method may be use to display information about a trailer coupler or guide a vehicle to line up a vehicle hitch to a trailer coupler.

Method and apparatus for providing trailer information

Self-watering planter

Pasta

Growing community of inventors

Pittsburgh, PA, United States of America

Joseph Edward Roth