Disney Enterprises, Inc.

Eidgenossische Technische Hochschule Zurich

Eth Zurich

Eth Züruch (Eidgenössische Technische Hochschule Zürich)

Yagiz Aksoy

Tunc Ozan Aydin

Ahmet Cengiz Öztireli

Jingwei Tang

Aljosa A Smolic

Marc Pollefeys

A method of generating a training data set for training an image matting machine learning model includes receiving a plurality of foreground images, generating a plurality of composited foreground images by compositing randomly selected foreground images from the plurality of foreground images, and generating a plurality of training images by compositing each composited foreground image with a randomly selected background image. The training data set includes the plurality of training images.

Learning-based sampling for image matting

Techniques are disclosed for image matting. In particular, embodiments decompose the matting problem of estimating foreground opacity into the targeted subproblems of estimating a background using a first trained neural network, estimating a foreground using a second neural network and the estimated background as one of the inputs into the second neural network, and estimating an alpha matte using a third neural network and the estimated background and foreground as two of the inputs into the third neural network. Such a decomposition is in contrast to traditional sampling-based matting approaches that estimated foreground and background color pairs together directly for each pixel. By decomposing the matting problem into subproblems that are easier for a neural network to learn compared to traditional data-driven techniques for image matting, embodiments disclosed herein can produce better opacity estimates than such data-driven techniques as well as sampling-based and affinity-based matting approaches.

Embodiments can provide a strategy for controlling information flow both from known opacity regions to unknown regions, as well as within the unknown region itself. This strategy is formulated through the use and refinement of various affinity definitions. As a result of this strategy, a final linear system can be obtained, which can be solved in closed form. One embodiment pertains to identifying opacity information flows. The opacity information flow may include one or more of flows from pixels in the image that have similar colors to a target pixel, flows from pixels in the foreground and background to the target pixel, flows from pixels in the unknown opacity region in the image to the target pixel, flows from pixels immediately surrounding the target pixels in the image to the target pixel, and any other flow.

Designing effective inter-pixel information flow for natural image matting

This disclosure related to systems and methods of palette-based color editing. A set of principle colors may be identified from among individual colors of individual pixels of an image. Color distributions about the individual principle colors may be determined. An individual color distribution may include a homogenous range of colors that may vary with respect to an individual principle color. Pixels may be associated with one or more pixel groups based on correspondences between individual colors of the individual pixels and individual colors included in the individual color distributions corresponding to the individual pixel groups. Color editing may be effectuated by modifying colors of pixels in a given group independently from other colors of other pixels in other groups.

Growing community of inventors

Zurich, Switzerland

Yagiz Aksoy