Microsoft Technology Licensing, LLC

Suhit Sinha

Sumit Srivastava

Ananth Sankar

Rushi P Bhatt

Jidnya Samir Shah

Somya Gupta

Akshat Mathur

Sankha Subhra Mullick

Systems and techniques for are described herein. A content passage is received from a corpus of training data comprising labeled training data and unlabeled training data. A query is received from a query hierarchy for a classification domain. The content passage and the query are embedded to form a passage-query pair. A predicted result for the passage-query pair is generated based on a calculated probability of the predicted result being within an answer threshold. A passage-query-result triplet is generated that comprises the passage-query pair and the predicted result according to the query hierarchy for the classification domain. Vectors of the content classification large language model are updated using the passage-query-result triplet.

Zero-shot training for multimodal content classifier

Methods, systems, and computer programs are presented for detecting near duplicates and partial matches of videos. One method includes an operation for receiving a video containing frames. For each frame, keypoints are determined within the frame. For each keypoint, a horizontal gradient vector is calculated based on a horizontal gradient at the keypoint and a vertical gradient vector is calculated based on a vertical gradient at the keypoint. The horizontal and vertical gradients are binary vectors. Further, a keypoint description is generated for each keypoint based on the horizontal gradient vector and the vertical gradient vector. Further, the frames are matched to frames of videos in a video library based on the keypoint descriptions of the keypoints in the frame in the videos in the video library. Further, a determination is made if the video has near duplicates in the video library based on the matching.

Partial-video near-duplicate detection

Technologies for generating a multi-modal representation of an image based on the image content are provided. The disclosed techniques include receiving an image, to be classified, that comprises one or more embedded text characters. The one or more embedded text characters are identified from the image and a first machine learning model is used to generate a text vector that represents a numerical representation of the one or more embedded text characters. A second machine learning model is used to generate an image vector that represents a numerical representation of the graphical portion of the image. The text vector and the image vector are used as input to generate a multi-modal vector that contains information from both the text vector and the image vector. The image may be classified into one of a plurality of image classifications based upon the information in the multi-modal vector.

Generating multi modal image representation for an image

Growing community of inventors

West Bengal, India

Suhit Sinha