Microsoft Technology Licensing, LLC

Di Wen

Ananth Sankar

Vipin Gupta

Saurabh Kataria

Ying Xuan

Bharat Kumar Jain

Sakshi Verma

Xue Xia

Nikita Gupta

Bhargavkumar Kanubhai Patel

Xun Luan

Sirjan Kafle

Aman Gupta

Described herein are systems and methods for generating an embedding—a learned representation—for an image. The embedding for the image is derived to capture visual aspects, as well as textual aspects, of the image. An encoder-decoder is trained to generate the visual representation of the image. An optical character recognition (OCR) algorithm is used to identify text/words in the image. From these words, an embedding is derived by performing an average pooling operation on pre-trained embeddings that map to the identified words. Finally, the embedding representing the visual aspects of the image is combined with the embedding representing the textual aspects of the image to generate a final embedding for the image.

Utilizing visual and textual aspects of images with recommendation systems

Growing community of inventors

Sunnyvale, CA, United States of America

Di Wen