Google Inc.

Nvidia Corporation

Innfarn Yoo

Feng Yang

Xiyang Luo

Rohit Taneja

Example embodiments allow for training of encoders (e.g., artificial neural networks (ANNs)) to facilitate dithering of images that have been subject to quantization in order to reduce the number of colors and/or size of the images. Such a trained encoder generates a dithering image from an input quantized image that can be combined, by addition or by some other process, with the quantized image to result in a dithered output image that exhibits reduced banding or is otherwise aesthetically improved relative to the un-dithered quantized image. The use of a trained encoder to facilitate dithering of quantized images allows the dithering to be performed in a known period of time using a known amount of memory, in contrast to alternative iterative dithering methods. Additionally, the trained encoder can be differentiable, allowing it to be part of a deep learning image processing pipeline or other machine learning pipeline.

Encoders for improved image dithering

A computer-implemented method that provides watermark-based image reconstruction to compensate for lossy encoding schemes. The method can generate a difference image describing the data loss associated with encoding an image using a lossy encoding scheme. The difference image can be encoded as a message and embedded in the encoded image using a watermark and later extracted from the encoded image. The difference image can be added to the encoded image to reconstruct the original image. As an example, an input image encoded using a lossy JPEG compression scheme can be embedded with the lost data and later reconstructed, using the embedded data, to a fidelity level that is identical or substantially similar to the original.

Watermark-based image reconstruction

In various examples, a two-dimensional (2D) and three-dimensional (3D) deep neural network (DNN) is implemented to fuse 2D and 3D object detection results for classifying objects. For example, regions of interest (ROIs) and/or bounding shapes corresponding thereto may be determined using one or more region proposal networks (RPNs)—such as an image-based RPN and/or a depth-based RPN. Each ROI may be extended into a frustum in 3D world-space, and a point cloud may be filtered to include only points from within the frustum. The remaining points may be voxelated to generate a volume in 3D world space, and the volume may be applied to a 3D DNN to generate one or more vectors. The one or more vectors, in addition to one or more additional vectors generated using a 2D DNN processing image data, may be applied to a classifier network to generate a classification for an object.

Leveraging multidimensional sensor data for computationally efficient object detection for autonomous machine applications

Example embodiments allow for training of encoders (e.g., artificial neural networks (ANNs)) to generate a color palette based on an input image. The color palette can then be used to generate, using the input image, a quantized, reduced color depth image that corresponds to the input image. Differences between a plurality of such input images and corresponding quantized images are used to train the encoder. Encoders trained in this manner are especially suited for generating color palettes used to convert images into different reduced color depth image file formats. Such an encoder also has benefits, with respect to memory use and computational time or cost, relative to the median-cut algorithm or other methods for producing reduced color depth color palettes for images.

Deep palette prediction

In various examples, a two-dimensional (2D) and three-dimensional (3D) deep neural network (DNN) is implemented to fuse 2D and 3D object detection results for classifying objects. For example, regions of interest (ROIs) and/or bounding shapes corresponding thereto may be determined using one or more region proposal networks (RPNs)&#x2014;such as an image-based RPN and/or a depth-based RPN. Each ROI may be extended into a frustum in 3D world-space, and a point cloud may be filtered to include only points from within the frustum. The remaining points may be voxelated to generate a volume in 3D world space, and the volume may be applied to a 3D DNN to generate one or more vectors. The one or more vectors, in addition to one or more additional vectors generated using a 2D DNN processing image data, may be applied to a classifier network to generate a classification for an object.

Growing community of inventors

Fremont, CA, United States of America

Innfarn Yoo