Computer-implemented multi-scale machine learning model for the multiple channel output enhancement of compressed video

Abstract

The present disclosure relates to methods, apparatus, systems, and non-transitory computer-readable storage media for training and using a multi-scale machine learning model for the multiple channel output enhancement of compressed video. According to some examples, a computer-implemented method includes receiving a video comprising a frame at a content delivery service; performing an encode on the frame of the video by the content delivery service that coverts the frame from a pixel domain to a transform domain and back to the pixel domain to generate first pixel values and first sample values based on a prediction for a block of the frame at a first resolution; generating a first set of features at the first resolution, by a machine learning model of the content delivery service, for a first input at the first resolution of the first pixel values and the first sample values of the block; generating a second set of features at a second lower resolution than the first resolution, by the machine learning model of the content delivery service, for a second input based on the first pixel values and the first sample values of the block; upsampling the second set of features to the first resolution to generate an upsampled second set of features; generating, by the machine learning model of the content delivery service, a multiple channel output based on the first set of features and the upsampled second set of features; generating a modified version of the frame based on the encode; generating a correction value based on the multiple channel output; and transmitting the modified version of the frame and the correction value from the content delivery service to a viewer device.