Google Inc.

Filip Pavetic

King Hong Thomas Leung

Matthias Rochus Konrad

Dmitrii Tochilkin

Hanna Pasula

Pedro Gonnet Anders

Valerii Zamaraiev

Roman Vorushin

Mayank Kandpal

Bakhodir Ashirmatov

A system and methods are disclosed for using a trained machine learning model to identify constituent images within composite images. A method may include providing data identifying a first image as input to a machine learning model trained using training data identifying a plurality of composite images that each include one or more constituent images, and determining, using one or more outputs of the trained machine learning model, that the first image is a composite image that includes a first constituent image, wherein at least a portion of the first constituent image is in a spatial area of the first image, and wherein the first constituent image corresponds to a frame of a video embedded into the first image.

Using machine learning to detect which part of the screen includes embedded frames of an uploaded video

Video content screening using a trained video screening model trained using self-supervised training includes automatically generating a training dataset by obtaining predicate screening data indicating a predicate temporal segment within a training video and a corresponding reference temporal segment within the reference video, obtaining candidate screening data for an extended temporal segment from the training video, wherein the extended temporal segment includes the predicate temporal segment and at least one frame from the training video adjacent to the predicate temporal segment, wherein the candidate screening data indicates a similarity between a screening frame from the reference video and a spatial portion of a candidate frame from the extended temporal segment, and, in response to a determination that a determined similarity between the candidate subframe including, in the automatically generated training dataset, training example data indicating the similarity between the candidate subframe and the screening frame.

Video screening using a machine learning video screening model trained using self-supervised training

A system and methods are disclosed for using a trained machine learning model to identify constituent images within composite images. A method may include providing pixel data of a first image as input to the trained machine learning model, obtaining one or more outputs from the trained machine learning model, and extracting, from the one or more outputs, an indication that the first image is a composite image that includes a constituent image, wherein at least a portion of the constituent image is in a spatial area of the first image.

Methods, systems, and media for analyzing spherical video content are provided. More particularly, methods, systems, and media for detecting two-dimensional videos placed on a sphere in abusive spherical video content are provided. In some embodiments, the method comprises: receiving an identifier of a spherical video content item, wherein the spherical video content item has a plurality of views; selecting a first frame of the spherical video content item; projecting the first frame of the spherical video content item to a two-dimensional region using a projection defined by a mapping according to which neighboring points of the first frame are mapped to respective neighboring points of the region, and one or more contiguous portions of the frame are each mapped to a corresponding plurality of contiguous portions of the region; identifying an area within the region which meets a criterion indicative of the region having a likelihood above a threshold of including a particular type of content; in response to identifying the area within the region which meets the criterion, analyzing the identified area of the region using a video fingerprinting technique; and, in response to determining that content associated with the identified area of the region matches a reference content item of a plurality of reference content items using the video fingerprinting technique, generating an indication of the match in association with the identifier of the spherical video content item.

Methods, systems, and media for detecting two-dimensional videos placed on a sphere in abusive spherical video content

Generating a training image for use in training a region-of-interest detector that is trained to detect regions-of-interest within images includes generating a closed geometric shape; filling the closed geometric shape with a filler to obtain a blob; overlaying the blob on an edge of an image to obtain the training image, where the image includes a region-of-interest and a background region, and where the edge separates the region-of-interest from the background region; and using the training image to train the region-of-interest detector to detect a boundary of the region-of-interest. An input to the region-of-interest detector in a training phase includes the training image and a first indication of coordinates of the region-of-interest in the training image. An output of the region-of-interest detector includes a second indication of an area of the training image and a probability of the area of the training image being the region-of-interest.

Randomly generated blobs to improve object-detection training for framed video content

A system and methods are disclosed for using a trained machine learning model to identify constituent images within composite images. A method may include providing pixel data of a first image as input to the trained machine learning model, obtaining one or more outputs from the trained machine learning model, and extracting, from the one or more outputs, a level of confidence that (i) the first image is a composite image that includes a constituent image, and (ii) at least a portion of the constituent image is in a particular spatial area of the first image.

Methods, systems, and media for analyzing spherical video content are provided. More particularly, methods, systems, and media for detecting two-dimensional videos placed on a sphere in abusive spherical video content by tiling the sphere are provided. In some embodiments, the method comprises: receiving an identifier of a spherical video content item, wherein the spherical video content item has a plurality of views and wherein the spherical video content item is encoded into a plurality of two-dimensional video frames; selecting a first frame of the plurality of two-dimensional video frames associated with the spherical video content item; dividing the first frame into a plurality of tiles spanning the first frame of the spherical video content item; calculating, for each tile of the plurality of tiles, a probability that the tile includes content of a particular type of content; determining, for each tile, whether the probability exceeds a predetermined threshold; in response to determining, for a particular tile, that the probability exceeds the predetermined threshold, causing the content associated with the tile to be analyzed using a video fingerprinting technique; and in response to determining, using the video fingerprinting technique, that the content associated with the tile matches a reference content item of a plurality of reference content items, generating an indication of the match in association with the identifier of the spherical video content item.

Methods, systems, and media for detecting two-dimensional videos placed on a sphere in abusive spherical video content by tiling the sphere

Methods, systems, and media for detecting and transforming rotated video content items are provided. The method comprises: receiving a video having a plurality of frames, wherein the video is associated with a first fingerprint; determining a rotation value associated with at least a portion of the plurality of frames to obtain a plurality of rotation values; determining an overall rotation value associated with the video based on a portion of the plurality of rotation values; determining whether at least one additional fingerprint of the video should be generated based on the overall rotation value; in response to determining that the at least one additional fingerprint of the video should be generated based on the overall rotation value, selecting a rotation transform based on the overall rotation value that rotates the plurality of frames of the video to an initial rotation position; applying the rotation transform to at least a portion of the plurality of frames of the video; generating a second fingerprint that represents the transformed video; and comparing the second fingerprint of the transformed video to a plurality of fingerprints associated with reference videos to determine whether the video corresponding to the transformed video matches one of the reference videos.

Methods, systems, and media for detecting and transforming rotated video content items

Fingerprinting a video including video frames is disclosed. A method includes receiving the video, generating sub-images, generating sub-fingerprints for the video using the sub-images, and matching the video to a reference video using the sub-fingerprints. Generating sub-images includes, for a video frame of some of the video frames, generating a binary image for the video frame, identifying a first region of the binary image, and identifying a sub-image of the video frame that is co-extensive and co-located with the first region of the binary image. A pixel of the video frame is identified in the binary image by a first value or a second value where the first value indicates a motion pixel and the second value indicates a still pixel. The first region includes more of the first value than the second value, and the first region is indicative of a motion in the video frame.

Detecting multiple parts of a screen to fingerprint to detect abusive uploading videos

A system and methods are disclosed for training a machine learning model to identify constituent images within composite images. In one implementation, a composite image is generated, where the composite image comprises a first portion containing pixel data of a first constituent image, and a second portion containing pixel data of a second constituent image. A first training input comprising pixel data of the composite image and a first target output for the first training input are generated, where the first target output identifies a position of the first portion within the composite image. The training data is provided to train the machine learning model on (i) a set of training inputs comprising the first training input and (ii) a set of target outputs comprising the first target output.

Methods, systems, and media for identifying content in stereoscopic videos and, more particularly, for detecting abusive stereoscopic videos by generating fingerprints for multiple portions of a video frame are provided. The method comprises: receiving, from a user device, a video content item for uploading to a content provider; selecting a frame from a plurality of frames of the video content item for generating one or more fingerprints corresponding to the video content item; generating a first fingerprint corresponding to the selected frame, a second fingerprint corresponding to a first encoded portion of the selected frame, and a third fingerprint corresponding to a second encoded portion of the selected frame; comparing each of the first fingerprint, the second fingerprint, and the third fingerprint to a plurality of reference fingerprints corresponding to reference video content items; determining whether at least one of the first fingerprint, the second fingerprint, and the third fingerprint match a reference fingerprint of the plurality of reference fingerprints; and, in response to determining that at least one of the first fingerprint, the second fingerprint, and the third fingerprint match the reference fingerprint, causing an indication of the match to be presented on the user device.

Methods, systems, and media for detecting abusive stereoscopic videos by generating fingerprints for multiple portions of a video frame

A method of identifying similar media items is described. The method include identifying a first multiplicity of fingerprints representative of content segments of variable duration for a first media item and a second multiplicity of fingerprints representative of content segments of variable duration for a second media item. The method further includes comparing, by a processing device, a first group of the first multiplicity of fingerprints to a second group of the second multiplicity of fingerprints to generate a first similarity score indicative of a similarity between the first group of fingerprints and the second group of fingerprints. The method also includes determining an alignment score for the first multiplicity of fingerprints and the second multiplicity of fingerprints using the first similarity score.

Multi-step sequence alignment

Whether a not a given video includes video content of other videos is determined by evaluating video content as well as video content transitions over time of videos. One or more fingerprints of the video are created and used to match against reference fingerprints of reference videos to identify candidate videos that is likely to have matching video content as an input video. After a candidate video is identified, a likelihood of the candidate video and an input video having matching video content is evaluated. A confidence score based on the evaluation of video content as well as video content transitions over time for a pair of videos including the input video and a candidate video is generated. When the confidence score exceeds a threshold score, it is determined that the pair of videos is likely to include the matching video content.

Growing community of inventors

Zurich, Switzerland

Filip Pavetic