Google Inc.

Xiyang Luo

Feng Yang

Innfarn Yoo

Dake He

Huiwen Chang

Ryan Matthew Haggarty

Roza Chojnacka

Elnaz Barshan Tashnizi

Jamieson Robert Kerns

Meltem Oktem

Ce Liu

Peyman Milanfar

Ondrej Stava

Tianhao Zhang

Archana Kannan

Nada Elassal

Michael Gene Goebel

Rajan Patel

Junjie Ke

Elad Edwin Tzvi Eban

Ruohan Zhan

Catherine Shyu

Luying Li

Chao-Hung Chen

Wenjing Kang

Shun-Chuan Chen

Alan Mackey

Yicong Tian

Uday Idnani

Xia Li

Example embodiments allow for training of encoders (e.g., artificial neural networks (ANNs)) to facilitate dithering of images that have been subject to quantization in order to reduce the number of colors and/or size of the images. Such a trained encoder generates a dithering image from an input quantized image that can be combined, by addition or by some other process, with the quantized image to result in a dithered output image that exhibits reduced banding or is otherwise aesthetically improved relative to the un-dithered quantized image. The use of a trained encoder to facilitate dithering of quantized images allows the dithering to be performed in a known period of time using a known amount of memory, in contrast to alternative iterative dithering methods. Additionally, the trained encoder can be differentiable, allowing it to be part of a deep learning image processing pipeline or other machine learning pipeline.

Encoders for improved image dithering

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for determining a visually imperceptible or a visually perceptible watermark and outputting a result based on the determination. A watermark decoder receives an input image. The watermark decoder applies a decoder machine learning model to decode a watermarks at different levels of zoom. The water mark decoder determines whether a watermark was decoded to obtain a decoded watermark. The watermark decoder outputs a result based on the determination whether the watermark was decoded through application of the decoder machine learning model to the input image that includes outputting a zoomed output decoded through application of the decoder machine learning model to the input image.

Zoom agnostic watermark extraction

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for detecting and decoding a visually imperceptible or perceptible watermark. A watermark detection apparatus determines whether the particular image includes a visually imperceptible or perceptible watermark using detector a machine learning model. If the watermark detection apparatus detects a watermark, the particular image is routed to a watermark decoder. If the watermark detection apparatus cannot detect a watermark in the particular image, the particular image is filtered from further processing. The watermark decoder decodes the visually imperceptible or perceptible watermark detected in the particular image. After decoding, an item depicted in the particular image is validated based data extracted from the decoded visually imperceptible or perceptible watermark.

Item validation based on visually imperceptible waterpark decoding

Systems, devices, methods, and computer readable medium for evaluating visual quality of digital content are disclosed. Methods can include training machine learning models on images. A request is received to evaluate quality of an image included in a current version of a digital component generated by the computing device. The machine learning models are deployed on the image to generate a score for each quality characteristic of the image. A weight is assigned to each score to generate weighted scores. The weighted scores are combined to generate a combined score for the image. The combined score is compared to one or more thresholds to generate a quality of the image.

Evaluating visual quality of digital content

A computer-implemented method that provides watermark-based image reconstruction to compensate for lossy encoding schemes. The method can generate a difference image describing the data loss associated with encoding an image using a lossy encoding scheme. The difference image can be encoded as a message and embedded in the encoded image using a watermark and later extracted from the encoded image. The difference image can be added to the encoded image to reconstruct the original image. As an example, an input image encoded using a lossy JPEG compression scheme can be embedded with the lost data and later reconstructed, using the embedded data, to a fidelity level that is identical or substantially similar to the original.

Watermark-based image reconstruction

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for jointly training an encoder that generates a watermark and a decoder that decodes a data item encoded within the watermark. The training comprises obtaining a plurality of training images and data items. For each training image, a first watermark is generated using an encoder and a subsequent second watermark is generated by tiling two or more first watermarks. The training image is watermarked using the second watermark to generate a first error value and distortions are added to the watermarked image. A distortion detector predicts the distortions based on which the distorted image is modified. The modified image is decoded by the decoder to generate a predicted data item and a second error value. The training parameters of the encoder and decoder are adjusted based on the first and the second error value.

End-to-end watermarking system

Example embodiments allow for training of encoders (e.g., artificial neural networks (ANNs)) to generate a color palette based on an input image. The color palette can then be used to generate, using the input image, a quantized, reduced color depth image that corresponds to the input image. Differences between a plurality of such input images and corresponding quantized images are used to train the encoder. Encoders trained in this manner are especially suited for generating color palettes used to convert images into different reduced color depth image file formats. Such an encoder also has benefits, with respect to memory use and computational time or cost, relative to the median-cut algorithm or other methods for producing reduced color depth color palettes for images.

Deep palette prediction

Methods, systems, and computer programs encoded on a computer storage medium, that relate to extracting digital watermarks from images, irrespective of distortions introduced into these images. Methods can include inputting a first data item into a channel encoder that can generate a first encoded data item that is greater in length than the first data item and that (1) includes the input data item and (2) new data this is redundant of the input data item. Based on the first encoded data item and a first image, an encoder model can generate a first encoded image into which the first encoded data is embedded as a digital watermark. A decoder model can decode the first encoded data item to generate a second data, which can be decoded by the channel decoder to generate data that is predicted to be the first data.

Image watermarking

A method can perform a process with a method including capturing an image, determining an environment that a user is operating a computing device, detecting a hand gesture based on an object in the image, determining, using a machine learned model, an intent of a user based on the hand gesture and the environment, and executing a task based at least on the determined intent.

Intent detection with a computing device

Example aspects of the present disclosure are directed to systems and methods for learning classification models which satisfy constraints such as, for example, constraints that can be expressed as a predicted positive rate or negative rate on a subset of the training dataset. In particular, through the use of quantile estimators, the systems and methods of the present disclosure can transform a constrained optimization problem into an unconstrained optimization problem that is solved more efficiently and generally than the constrained optimization problem. As one example, the unconstrained optimization problem can include optimizing an objective function where a decision threshold of the classification model is expressed as an estimator of a quantile function on the classification scores of the machine-learned classification model for a subset of the training dataset at a desired quantile.

Constrained classification and ranking via quantiles

A system and method may provide for spatial and semantic auto-completion of an augmented or mixed reality environment. The system may detect physical objects in a physical environment based on analysis of image frames captured by an image sensor of a computing device. The system may detect spaces in the physical environment that are occupied by the detected physical objects, and may detect spaces that are unoccupied in the physical environment. Based on the identification of the detected physical objects, the system may gain a semantic understanding of the physical environment, and may determine suggested objects for placement in the physical environment based on the semantic understanding. The system may place virtual representations of the suggested objects in a mixed reality scene of the physical environment for user consideration.

Growing community of inventors

Sunnlyvale, CA, United States of America

Xiyang Luo