Adobe, Inc.

Samsung Electronics Co., Ltd.

Korea Advanced Institute of Science and Technology

Sanghyun Woo

Dahun Kim

Joon-Young Lee

Sungjin Kim

Kyungsu Kim

In-So Kweon

Joonyoung Lee

Seoung Wug Oh

Kwanyong Park

Yunjae Jung

Embodiments are disclosed for performing per-clip object segmentation of objects in a video sequence using machine learning. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving a query video sequence and memory data, the memory data including a memory video frame from the query video sequence and an annotated memory video frame including an object mask for an object in the memory video frame, segmenting the query video sequence into a plurality of query video clips and passing a first set of query video frames of a first query video clip and the memory data through a trained encoder-decoder network, predicting a modified set of query video frames, including predictions of object masks for the object, and updating the memory data to include one or more frames of the first set of query video frames and the modified set of query video frames.

Per-clip video object segmentation using machine learning

An electronic device and a method thereof are provided. The electronic device includes: a memory and a processor configured to: acquire feature data of a plurality of images in a video using a first artificial neural network of a first artificial intelligence model; acquire a plurality of key frames of the video based on the feature data of the plurality of images using a second artificial neural network of the first artificial intelligence model; acquire first feature data of remaining key frames excluding at least one of the plurality of key frames using a first artificial neural network of a second artificial intelligence model; acquire second feature data including information about relationships between the remaining key frames based on the first feature data using a second artificial neural network of the second artificial intelligence model; and acquire texts for the plurality of key frames based on the second feature data.

Electronic device for key frame analysis and control method thereof

Systems and methods for panoptic video segmentation are described. A method may include identifying a target frame and a reference frame from a video, generating target features for the target frame and reference features for the reference frame, combining the target features and the reference features to produce fused features for the target frame, generating a feature matrix comprising a correspondence between objects from the reference features and objects from the fused features; and generating panoptic segmentation information for the target frame based on the feature matrix.

Video panoptic segmentation

A method, apparatus, non-transitory computer readable medium, and system for panoptic segmentation are described. Embodiments may generate a feature pyramid for an input image, wherein the feature pyramid comprises a plurality of feature maps at different resolution levels, apply an attention module to the feature pyramid to produce an enhanced feature map, combine the enhanced feature map with each of the plurality of feature maps to produce an enhanced feature pyramid, generate semantic information for the input image based on the enhanced feature pyramid, generate a plurality of object regions based on the enhanced feature pyramid, generate instance information for each of the plurality of object regions, and generate panoptic segmentation information for the input image based on the semantic information and the instance information for each of the plurality of object regions.

Panoptic segmentation

Growing community of inventors

Daejeon, South Korea

Sanghyun Woo