Adobe, Inc.

Ali Aminian

Kshitiz Garg

Aashish Misraa

Aseem Agarwala

Ajinkya Gorakhnath Kale

Pranav Vineet Aggarwal

Han Guo

Viswanathan Swaminathan

Ryan A Rossi

Saayan Mitra

Xue Bai

Feng Liu

Somdeb Sarkhel

Xiang Chen

William L Marino

Jui-hsien Wang

Swapneel Mehta

Gabriela Duncombe

Lakshya Lnu

Zhe Wang

Gunjan Aggarwal

Shivam Nalin Patel

Embodiments are disclosed for generating a temporally coherent video extension. The method includes displaying, on a graphical user interface, a user interface element representing a video to be extended, where the video includes a number of frames. The method further includes receiving an input via the graphical user interface associated with the user interface element. The input causes a visual change to the user interface element which represents a duration of an extension to be made to the video. The method further includes generating frames based on the duration of the extension. The generated frames use motion information determined from frames of the video. The motion information represents a per-pixel motion between at least a pair of frames of the video. The method further includes providing, for display on the graphical user interface, an extended video including the frames of the video and the generated frames.

Time domain video extension

Embodiments of the present invention provide systems, methods, and computer storage media for extracting moments of interest (e.g., video frames, video segments) from a video. In an example embodiment, independent and/or orthogonal machine learning models are used to extract different types of features considering different modalities, and each frame in the video is assigned an importance score for each model. The importance scores for each model are combined into an aggregated importance score for each frame in the video. Depending on the embodiment, the aggregated importance scores are used to visualize the score per frame, identify moments of interest, automatically crop down the video into a highlight reel, browse or visualize the moments of interest within the video, and/or search across multiple videos.

Customizable framework to extract moments of interest

Some aspects of the technology described herein perform identity identification on faces in a video. Object tracking is performed on detected faces in frames of a video to generate tracklets. Each tracklet comprises a sequence of consecutive frames in which each frame includes a detected face for a person. The tracklets are clustered using face feature vectors for detected faces of each tracklet to generate a plurality of clusters. Information is stored in an identity datastore, including a first identifier for a first identity in association with an indication of frames from tracklets in a first cluster from the plurality of clusters.

Tracking unique face identities in videos

Technology is disclosed herein for enhanced similarity search. In an implementation, a search environment includes one or more computing hardware, software, and/or firmware components in support of enhanced similarity search. The one or more components identify a modality for a similarity search with respect to a query object. The components generate an embedding for the query object based on the modality and based on connections between the query object and neighboring nodes in a graph. The embedding for the query object provides the basis for the search for similar objects.

Unified framework for multi-modal similarity search

Embodiments are disclosed for receiving a user input and an input video comprising multiple frames. The method may include extracting a text feature from the user input. The method may further include extracting a plurality of image features from the frames. The method may further include identifying one or more keyframes from the frames that include the object. The method may further include clustering one or more groups of the one or more keyframes. The method may further include generating a plurality of segmentation masks for each group. The method may further include determining a set of reference masks corresponding to the user input and the object. The method may further include generating a set of fusion masks by combining the plurality of segmentation masks and the set of reference masks. The method may further include propagating the set of fusion masks and outputting a final set of masks.

Text-based framework for video object selection

Techniques for recommending hashtags, including trending hashtags, are disclosed. An example method includes accessing a graph. The graph includes video nodes representing videos, historical hashtag nodes representing historical hashtags, and edges indicating associations among the video nodes and the historical hashtag nodes. A trending hashtag is identified. An edge is added to the graph between a historical hashtag node representing a historical hashtag and a trending hashtag node representing the trending hashtag, based on a semantic similarity between the historical hashtag and the trending hashtag. A new video node representing a new video is added to the video nodes of the graph. A graph neural network (GNN) is applied to the graph, and the GNN predicts a new edge between the trending hashtag node and the new video node. The trending hashtag is recommended for the new video based on prediction of the new edge.

Open-domain trending hashtag recommendations

Growing community of inventors

San Jose, CA, United States of America

Ali Aminian