Virtual camera system

Abstract

A virtual camera system comprises a plurality of physical cameras and a hardware setup miming software to create virtual viewpoints for the virtual camera system. The position of the physical cameras is constrained, where the main constraint is the overlap between the physical cameras. The present invention provides a method for creating a virtual viewpoint of a plurality of images captured by the plurality of cameras, the images comprising current frames and previous frames. The method comprises creating background frames by combining a set of previous frames; creating a plurality of foreground masks by performing segmentation on the plurality of current frames, the segmentation comprising separating foreground objects from background objects using the created background frames; extracting the foreground objects by AND-ing the plurality of current frames with the created plurality of foreground masks; rendering the foreground objects by using a three dimensional depth-box constrained viewpoint interpolation algorithm on the plurality of current frames, the three dimensional depth box being constructed for each foreground object by exploiting information from the created foreground masks and positioning information of the foreground objects; rendering a view-dependent background image by performing homographic transformations on the created background frames; and creating the virtual viewpoint by superimposing the rendered foreground objects on the rendered background frame.