Deep modular systems for image restoration

Abstract

Image processing systems and image processing techniques leveraging neural networks (e.g., convolutional neural networks (CNNs)) for image restoration tasks (e.g., for demosaicing tasks) are described. In certain aspects, Mixture of Experts (MoE) techniques may be employed, where multiple different expert networks are used to divide a problem space (e.g., image reconstruction tasks) into homogenous regions. For example, each MoE module may reconstruct a certain problem in an image, and a gating component may activate certain MoE modules to provide a reconstructed image. In some aspects, training and optimization techniques are described for each expert of the MoE architecture, to increase individual performance (e.g., a sub-task for each expert of an image processing system may be imposed in a residual manner, a gating function may be trained, etc.). Accordingly, image processing systems may leverage MoE architectures to support a large number of neural network parameters for improved image reconstruction applications.