Computer-implemented multi-scale machine learning model for the enhancement of compressed video

Abstract

The present disclosure relates to methods, apparatus, systems, and non-transitory computer-readable storage media for training and using a multi-scale machine learning model for the enhancement of compressed video. According to some examples, a computer-implemented method includes receiving a video at a content delivery service; performing an encode on a frame of the video by the content delivery service that converts the frame from a pixel domain to a transform domain and back to the pixel domain to generate first pixel values and a first residual for a block of the frame at a first resolution; generating a first set of features, by a machine learning model of the content delivery service, for an input, at a first resolution, of the first pixel values and the first residual of the block; generating a second set of features, by the machine learning model of the content delivery service, for an input, at a second lower resolution, of second pixel values and a second residual of the block; upsampling the second set of features to the first resolution to generate an upsampled second set of features; generating a modified version of the frame based on the first set of features and the upsampled second set of features; and transmitting the modified version of the frame to a frame buffer or from the content delivery service to a viewer device.