Microsoft Technology Licensing, LLC

Nnaisense Sa

Rupesh Kumar Srivastava

Li Deng

John Carlton Platt

Jianfeng Gao

Xiaodong He

Geoffrey G Zweig

Margaret Ann Mitchell

Saurabh Gupta

Forrest Iandola

Klaus Greff

Hao Fang

A computer-based method includes receiving an input signal at a neuron in a computer-based neural network that includes a plurality of neuron layers, applying a first non-linear transform to the input signal at the neuron to produce a plain signal, and calculating a weighted sum of a first component of the input signal and the plain signal at the neuron. In a typical implementation, the first non-linear transform is a function of the first component of the input signal and at least a second component of the input signal.

Highly trainable neural network configuration

Disclosed herein are technologies directed to discovering semantic similarities between images and text, which can include performing image search using a textual query, performing text search using an image as a query, and/or generating captions for images using a caption generator. A semantic similarity framework can include a caption generator and can be based on a deep multimodal similar model. The deep multimodal similarity model can receive sentences and determine the relevancy of the sentences based on similarity of text vectors generated for one or more sentences to an image vector generated for an image. The text vectors and the image vector can be mapped in a semantic space, and their relevance can be determined based at least in part on the mapping. The sentence associated with the text vector determined to be the most relevant can be output as a caption for the image.

Discovery of semantic similarities between images and text

Growing community of inventors

Lugano, Switzerland

Rupesh Kumar Srivastava