Unity Technologies Sf

Weta Digital Limited

Josh Hardgrave

Dejan Momcilovic

Erik B Edlund

Tobias B Schmidt

Embodiments allow live action images from an image capture device to be composited with computer generated images in real-time or near real-time. The two types of images (live action and computer generated) are composited accurately by using a depth map. In an embodiment, the depth map includes a 'depth value' for each pixel in the live action image. In an embodiment, steps of one or more of feature extraction, matching, filtering or refinement can be implemented, at least in part, with an artificial intelligence (AI) computing approach using a deep neural network with training. A combination of computer-generated ('synthetic') and live-action (&#x201c;recorded&#x201d;) training data is created and used to train the network so that it can improve the accuracy or usefulness of a depth map so that compositing can be improved.

System for image compositing including training with synthetic data

Embodiments allow camera effects, such as imaging noise, to be included in a generation of a synthetic data set for use in training an artificial intelligence model to produce an image depth map. The image depth map can then be employed to assist in compositing live action images from an image capture device with computer generated images in real-time or near real-time. The two types of images (live action and computer generated) are composited accurately by using a depth map. In an embodiment, the depth map includes a 'depth value' for each pixel in the live action image. In an embodiment, steps of one or more of feature extraction, matching, filtering or refinement can be implemented, at least in part, with an artificial intelligence (AI) computing approach using a deep neural network with training.

Growing community of inventors

Wellington, New Zealand

Josh Hardgrave