International Business Machines Corporation

Shi Hui Gui

Li Zhou

Ting Chen

Jian Jun Wang

Xiao Xia Mao

Wei Jun Zheng

Xiao Feng Ji

Jing Xia

Yidan Lei

A method, computer system, and a computer program product are provided for training a neural network for finding queried videos. Two pairs of video clips and associated text are obtained from a first dataset and a second dataset. The first dataset is used to train two video encoders by providing the video clips to the encoders as input and providing the outputs to a cosine similarity calculator. The second dataset is used to train a multi-mentor paradigm with two mentors. A first mentor and a second mentor are each provided the pair of textual data inputs. The first mentor provides a similarity value comparison, and the second mentor provides a word mover distance. Using the output from the multi-mentor paradigm and the encoders, a contrastive loss is calculated and used to provide contrastive learning of video features by differentiating similarity and dissimilarity of the video clips.

Video retrieval techniques using video contrastive learning

According to an embodiment, a source object presented in a source video is identified. Attribute information of the source object in respective frames of a sequence of source frames in the source video is identified. The attribute information represents an animation effect associated with the source object across the sequence of source frames. The attribute information is provided for use in reproducing the animation effect in a target video.

Animation effect reproduction

Growing community of inventors

Shanghai, China

Shi Hui Gui