Joint unsupervised object segmentation and inpainting

Abstract

The invention relates to implementation of image processing functions associated with finding the boundaries of objects, removing objects from an image, inserting objects into an image, creating new images from a combination of existing images. Proposed is a method for automated image processing and a computing system for performing automated image processing, comprising: first neural network for forming a coarse image z by segmenting an object O from an original image x containing the object O and background Bby a segmentation mask, and, using the mask, cutting off the segmented object O from the image x and pasting it onto an image y containing only background B, second neural network for constructing an enhanced version of an image ŷ with pasted segmented object O by enhancing coarse image z based on the original images {circumflex over (x)} and y and the mask m; third neural network, for restoring the background-only image {circumflex over (x)} without removed segmented object O by inpainting image obtained by zeroing out pixels of image x using the mask m; wherein the first, second and third neural networks are combined into common architecture of neural network for sequential performing segmentation, enhancing and inpainting and for simultaneously learning, wherein the common architecture of neural network accepts the images and outputs processed images of the same dimensions.