Content-adaptive online training for dnn-based cross component prediction with scaling factors

Abstract

A method and apparatus for neural network based cross component prediction with scaling factors during encoding or decoding of an image frame or a video sequence, which may include training a deep neural network (DNN) cross component prediction (CCP) model with at least one or more scaling factors, wherein the at least one or more scaling factors are learned by optimizing a rate-distortion loss based on an input video sequence comprising a luma component, and reconstructing a chroma component based on the luma component using the trained DNN CCP model with the at least one or more scaling factors for chroma prediction. The trained DNN CCP may be updated for chroma prediction of the input video sequence using the one or more scaling factors, and performing chroma prediction of the input video sequence using the updated DNN CCP model with the one or more scaling factors.