Waveone Inc.

Carissa Lew

Lubomir Dimitrov Bourdev

Oren Rippel

Sanjay Nair

Alexander Anderson

Steve Branson

Steven Branson

The compression system trains a machine-learned encoder and decoder through an autoencoder architecture. The encoder can be deployed by a sender system to encode content for transmission to a receiver system, and the decoder can be deployed by the receiver system to decode the encoded content and reconstruct the original content. The encoder is coupled to receive content and output a tensor as a compact representation of the content. The content may be, for example, images, videos, or text. The decoder is coupled to receive a tensor representing content and output a reconstructed version of the content. The compression system trains the autoencoder with a discriminator to reduce compression artifacts in the reconstructed content. The discriminator is coupled to receive one or more input content, and output a discrimination prediction that discriminates whether the input content is the original or reconstructed version of the content.

Using generative adversarial networks in compression

A machine learning (ML) task system trains a neural network model that learns a compressed representation of acquired data and performs a ML task using the compressed representation. The neural network model is trained to generate a compressed representation that balances the objectives of achieving a target codelength and achieving a high accuracy of the output of the performed ML task. During deployment, an encoder portion and a task portion of the neural network model are separately deployed. A first system acquires data, applies the encoder portion to generate a compressed representation, performs an encoding process to generate compressed codes, and transmits the compressed codes. A second system regenerates the compressed representation from the compressed codes and applies the task model to determine the output of a ML task.

Data compression for machine learning tasks

An encoder system trains a compression model that includes an autoencoder model and a frame extractor model. The encoding portion of the autoencoder is coupled to receive a set of target frames and a previous state tensor for the set of target frames and generate compressed code. The decoding portion of the autoencoder is coupled to receive the compressed code and the previous state tensor for the set of frames and generate a next state tensor for the set of target frames. The frame extractor model is coupled to receive the next state tensor and generate a set of reconstructed frames that correspond to the set of target frames by performing one or more operations on the state tensor. The state tensor for the set of frames includes information from frames of the video that can be used by the frame extractor to generate reconstructed frames.

Machine-learning based video compression

A compression system includes an encoder and a decoder. The encoder can be deployed by a sender system to encode a tensor for transmission to a receiver system, and the decoder can be deployed by the receiver system to decode and reconstruct the encoded tensor. The encoder receives a tensor for compression. The encoder also receives a quantization mask and probability data associated with the tensor. Each element of the tensor is quantized using an alphabet size allocated to that element by the quantization mask data. The encoder compresses the tensor by entropy coding each element using the probability data and alphabet size associated with the element. The decoder receives the quantization mask data, the probability data, and the compressed tensor data. The quantization mask and probabilities are used to entropy decode and subsequently reconstruct the tensor.

Adaptive quantization

An enhanced encoder system generates residual bitstreams representing additional image information that can be used by an image enhancement system to improve a low quality image. The enhanced encoder system upsamples a low quality image and compares the upsampled image to a true high quality image to determine image inaccuracies that arise due to the upsampling process. The enhanced encoder system encodes the information describing the image inaccuracies using a trained encoder model as the residual bitstream. The image enhancement system upsamples the same low quality image to obtain a prediction of a high quality image that can include image inaccuracies. Given the residual bitstream, the image enhancement system decodes the residual bitstream using a trained decoder model and uses the additional image information to improve the predicted high quality image. The image enhancement system can provide an improved, high quality image for display.

Autoencoding image residuals for improving upsampled images

A compression system trains a machine-learned encoder and decoder. The encoder can be deployed by a sender system to encode content for transmission to a receiver system, and the decoder can be deployed by the receiver system to decode the encoded content and reconstruct the original content. The encoder receives content and generates a tensor as a compact representation of the content. The content may be, for example, images, videos, or text. The decoder receives a tensor and generates a reconstructed version of the content. In one embodiment, the compression system trains one or more encoding components such that the encoder can adaptively encode different degrees of information for regions in the content that are associated with characteristic objects, such as human faces, texts, or buildings.

Growing community of inventors

San Jose, CA, United States of America

Carissa Lew