The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Feb. 20, 2024
Filed:
Nov. 18, 2019
Google Llc, Mountain View, CA (US);
Ariel Fuxman, Redwood City, CA (US);
Aleksei Timofeev, Mountain View, CA (US);
Zhen Li, Sunnyvale, CA (US);
Chun-Ta Lu, Sunnyvale, CA (US);
Manan Shah, Los Altos, CA (US);
Chen Sun, San Francisco, CA (US);
Krishnamurthy Viswanathan, Sunnyvale, CA (US);
Chao Jia, Sunnyvale, CA (US);
GOOGLE LLC, Mountain View, CA (US);
Abstract
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for realizing a multimodal image classifier. In an aspect, a method includes, for each image of a plurality of images: processing the image by a textual generator model to obtain a set of phrases that are descriptive of the content of the image, wherein each phrase is one or more terms, processing the set of phrases by a textual embedding model to obtain an embedding of predicted text for the image, and processing the image using an image embedding model to obtain an embedding of image pixels of the image. Then a multimodal image classifier is trained on the embeddings of predicted text for the images and the embeddings of image pixels for the images to produce, as output, labels of an output taxonomy to classify an image based on the image as input.