The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Dec. 05, 2017
Filed:
Aug. 28, 2015
Microsoft Technology Licensing, Llc, Redmond, WA (US);
Jianfeng Gao, Woodinville, WA (US);
Xiaodong He, Sammamish, WA (US);
Saurabh Gupta, Berkeley, CA (US);
Geoffrey G. Zweig, Sammamish, GA (US);
Forrest Iandola, Berkeley, CA (US);
Li Deng, Redmond, WA (US);
Hao Fang, Seattle, WA (US);
Margaret A. Mitchell, Seattle, WA (US);
John C. Platt, Bellevue, WA (US);
Rupesh Kumar Srivastava, Lugano, CH;
Microsoft Technology Licensing, LLC, Redmond, WA (US);
Abstract
Disclosed herein are technologies directed to discovering semantic similarities between images and text, which can include performing image search using a textual query, performing text search using an image as a query, and/or generating captions for images using a caption generator. A semantic similarity framework can include a caption generator and can be based on a deep multimodal similar model. The deep multimodal similarity model can receive sentences and determine the relevancy of the sentences based on similarity of text vectors generated for one or more sentences to an image vector generated for an image. The text vectors and the image vector can be mapped in a semantic space, and their relevance can be determined based at least in part on the mapping. The sentence associated with the text vector determined to be the most relevant can be output as a caption for the image.