Google Inc.

Junjie Ke

Feng Yang

Yilin Wang

Peyman Milanfar

Qifei Wang

Balineedu Adsumilli

Neil Birkbeck

Xiyang Luo

Hossein Talebi

Joong Yim

Xia Li

Catherine Shyu

Luying Li

Chao-Hung Chen

Wenjing Kang

Shun-Chuan Chen

Yicong Tian

Systems, devices, methods, and computer readable medium for evaluating visual quality of digital content are disclosed. Methods can include training machine learning models on images. A request is received to evaluate quality of an image included in a current version of a digital component generated by the computing device. The machine learning models are deployed on the image to generate a score for each quality characteristic of the image. A weight is assigned to each score to generate weighted scores. The weighted scores are combined to generate a combined score for the image. The combined score is compared to one or more thresholds to generate a quality of the image.

Evaluating visual quality of digital content

The technology employs a patch-based multi-scale Transformer () that is usable with various imaging applications. This avoids constraints on image fixed input size and predicts the quality effectively on a native resolution image. A native resolution image () is transformed into a multi-scale representation (), enabling the Transformer's self-attention mechanism to capture information on both fine-grained detailed patches and coarse-grained global patches. Spatial embedding () is employed to map patch positions to a fixed grid, in which patch locations at each scale are hashed to the same grid. A separate scale embedding () is employed to distinguish patches coming from different scales in the multiscale representation. Self-attention () is performed to create a final image representation. In some instances, prior to performing self-attention, the system may prepend a learnable classification token () to the set of input tokens.

Multi-scale transformer for image analysis

Methods, systems, and media for determining perceptual quality indicators of video content items are provided. In some embodiments, the method comprises: receiving a video content item; extracting a plurality of frames from the video content item; determining, using a first subnetwork of a deep neural network, a content quality indicator for each frame of the plurality of frames of the video content item; determining, using a second subnetwork of the deep neural network, a video distortion indicator for each frame of the plurality of frames of the video content item; determining, using a third subnetwork of the deep neural network, a compression sensitivity indicator for each frame of the plurality of frames of the video content item; generating a quality level for each frame of the plurality of frames of the video content item that concatenates the content quality indicator, the video distortion indicator, and the compression sensitivity indicator for that frame of the video content item; generating an overall quality level for video content item by aggregating the quality level of each frame of the plurality of frames; and causing a video recommendation to be presented based on the overall quality level of the video content item.

Methods, systems, and media for determining perceptual quality indicators of video content items

Growing community of inventors

East Palo Alto, CA, United States of America

Junjie Ke