Microsoft Technology Licensing, LLC

Bharat Kumar Jain

Smitkumar Narotambhai Marvaniya

Ananth Sankar

Vipin Gupta

Saurabh Kataria

Ying Xuan

Di Wen

Sakshi Verma

Xue Xia

Nikita Gupta

Xun Luan

Bhargavkumar Kanubhai Patel

Aman Gupta

Sirjan Kafle

Osaid Rehman Nasir

Technologies for language agnostic OCR extraction include identifying a word region of an image using optical character recognition, applying a language agnostic machine learning model to the word region, where the language agnostic machine learning model is trained on training data including a set of image-text pairs and a set of multilingual text translation pairs, receiving, from the language agnostic machine learning model, a word region embedding that is associated with the word region, searching a multilingual index for a text embedding that matches the word region embedding, receiving, from the multilingual index, text associated with the text embedding; and outputting at least one of the text or the text embedding to at least one downstream process, application, system, component, or network.

Language-agnostic OCR extraction

Described herein are systems and methods for generating an embedding—a learned representation—for an image. The embedding for the image is derived to capture visual aspects, as well as textual aspects, of the image. An encoder-decoder is trained to generate the visual representation of the image. An optical character recognition (OCR) algorithm is used to identify text/words in the image. From these words, an embedding is derived by performing an average pooling operation on pre-trained embeddings that map to the identified words. Finally, the embedding representing the visual aspects of the image is combined with the embedding representing the textual aspects of the image to generate a final embedding for the image.

Utilizing visual and textual aspects of images with recommendation systems

Growing community of inventors

Hyderabad Telangana, India

Bharat Kumar Jain