Adobe, Inc.

Seoung Wug Oh

Joon-Young Lee

Brian Lynn Price

John Nelson

Sanghyun Woo

Jaedong Hwang

Adam Pikielny

Kwanyong Park

Wujun Wang

Embodiments are disclosed for performing per-clip object segmentation of objects in a video sequence using machine learning. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a query video sequence and memory data, the memory data including a memory video frame from the query video sequence and an annotated memory video frame including an object mask for an object in the memory video frame, segmenting the query video sequence into a plurality of query video clips and passing a first set of query video frames of a first query video clip and the memory data through a trained encoder-decoder network, predicting a modified set of query video frames, including predictions of object masks for the object, and updating the memory data to include one or more frames of the first set of query video frames and the modified set of query video frames.

Per-clip video object segmentation using machine learning

Embodiments are disclosed for a machine learning-based chroma keying process. The method may include receiving an input including an image depicting a chroma key scene and a color value corresponding to a background color of the image. The method may further include generating a preprocessed image by concatenating the image and the color value. The method may further include providing the preprocessed image to a trained neural network. The method may further include generating, using the trained neural network, an alpha matte representation of the image based on the preprocessed image.

Machine learning-based chroma keying process

This disclosure describes one or more implementations of a panoptic segmentation system that generates panoptic segmented digital images that classify both known and unknown instances of digital images. For example, the panoptic segmentation system builds and utilizes a panoptic segmentation neural network to discover, cluster, and segment new unknown object subclasses for previously unknown object instances. In addition, the panoptic segmentation system can determine additional unknown object instances from additional digital images. Moreover, in some implementations, the panoptic segmentation system utilizes the newly generated unknown object subclasses to refine and tune the panoptic segmentation neural network to improve the detection of unknown object instances in input digital images.

Generating improved panoptic segmented digital images based on panoptic segmentation neural networks that utilize exemplar unknown object classes

Growing community of inventors

San Jose, CA, United States of America

Seoung Wug Oh