Microsoft Technology Licensing, LLC

Ricoh Company, Ltd.

Qifa Ke

Jonathan J Hull

Berna Erol

Michael Isard

Anitha Kannan

Jian Qiao Sun

Simon John Baker

Hidenobu Kishi

Yi Li

Manish Malik

Heung-Yeung Shum

Jorge Moraleda

Yinglian Xie

Zhong Wu

Dahua Lin

Rakesh Agrawal

Fang Yu

Ming Gang Liu

Eliot Carleton Gillum

Doran Chakraborty

Qi Liu

Emilio R Antunez

Partha Pratim Talukdar

Nikhil Rasiwasia

Jonathan R Tiao

Andrew Lookingbill

Jing Wang

Shipeng Li

Zhengyou Zhang

Peter E Hart

Matthew Uyttendaele

Martin Abadi

Yu Shi

Krishna C Vitaldevara

Li-wei He

Yuan Yu

Fang Wen

Jason D Walter

Jingdong Wang

Frank Torsten Bernd Seide

Rui Hu

Vijayan Prabhakaran

Srinivas Vadrevu

Yanfeng Sun

Xiaochuan Ni

Yu-ting Kuo

Krishnan Ramnath

Jia H Deng

Andreas Bode

Gang Luo

Rangan Majumder

Pushpraj Shukla

JunFeng Yang

Jingyue Wu

Yao Zhao

Oana Nicolov

Krishna Sravanthi Rajanala Sai

Dario Bigongiari

Rajanala Sai Krishna Sravanthi

Shigeharu Uda

Juliet Anne Bernstein

David Changsoo Lee

Jiarui Ren

Hidenobu c/o Ricoh Innovations Inc Kishi

Examples of the present disclosure describe systems and methods relating to generating relevance scores for one or more words of a passage which is an answer to a natural language query. For instance, a passage extracted from a highly relevant electronic file along with the query may encoded and augmented to generate a multi-dimensional, augmented semantic vectors using recurring neural networks. The augmented semantic vectors along with a multi-dimensional vector that represent words of the passage may be decoded to generate relevance scores for one or more words of the passage, based on levels of relevance to the query.

Method and system for highlighting answer phrases

Systems and techniques for sequence to sequence conversational query understanding are described herein. A query may be received that includes multiple words. It may be identified that the query is to be reformulated based on an attention value for an attention word in the query. Relationships may be determined among words of the query and words in a previously submitted query and words in results from the previously submitted query. The query may be reformulated based on the relationships. The reformulated query may be employed to retrieve query results.

Sequence to sequence conversational query understanding

Examples of the present disclosure describe systems and methods relating to generating a relevance score on a given natural language answer to a natural language query for ranking the answer among other answers for the query, while generating a summary passage and a likely query to the given passage. For instance, multi-layered, recurrent neural networks may be used to encode the query and the passage, along with a multi-layered neural network for information retrieval features, to generate a relevant score for the passage. A multi-layered, recurrent neural network with soft attention and sequence-to-sequence learning task may be used as a decoder to generate a summary passage. A common encoding neural network may be employed to encode the passage for the ranking and the summarizing, in order to present concise and accurate natural language answers to the query.

Method and system for ranking and summarizing natural language passages

Non-limiting examples of the present disclosure describe a unified ranking model that may be used by a plurality of entry points to return ranked results in response to received query data. The unified ranking model is provided as a service for a plurality of entry points. A query is received from an entry point of the plurality of entry points. Results data for the query data is retrieved. A unified ranking model is executed to rank the results data. Execution of the unified ranking model manipulates feature data of the unified ranking model based on user context signals associated with the received query data and acquired result retrieval signals corresponding with the retrieved results data. Execution of the unified ranking model generates ranked result data. Ranked results data is returned to the processing device corresponding with the entry point. Other examples are also described.

Single unified ranker

A Mixed Media Reality (MMR) system and associated techniques are disclosed. The MMR system provides mechanisms for forming a mixed media document that includes media of at least two types (e.g., printed paper as a first medium and digital content and/or web link as a second medium). The present invention provides systems, methods, and computer program products for modifying documents for shared use, and for collaborative discussion of shared documents. Captured digital images of documents or portions associated with a user are received, along with modifications to the images. Documents are recognized from the captured digital images, and the modifications to the images are applied to the documents. Alternatively, captured digital images of documents are received, and the documents are recognized along with a hotspot in the document. The user is authenticated to a website associated with the hotspot, and is provided access to an associated multimedia repository.

Indexed document modification sharing with mixed media reality

A summary of a document is generated in near real time. In aspects, an indication to summarize the document is received and the document is processed to generate a summary. For instance, processing includes extracting sentences from the document and generating a plurality of candidate passages from the extracted sentences. Features are extracted from each of the plurality of candidate passages and each candidate passage is ranked based at least in part on the extracted features. High-ranking candidate passages are considered likely to be important and/or representative of the document. A summary of the document is generated including one or more of the high-ranking candidate passages. The summary includes portions of the document that are considered important and/or representative of the document, so a user may review the summary in lieu of reading the entire document.

Automatic document summarization using search engine intelligence

A set of data points is divided into a plurality of subsets of data points. A set of cluster closures is generated based at least in part on the subset of data points. Each cluster closure envelopes a corresponding cluster of a set of clusters and is comprised of data points of the enveloped cluster and data points neighboring the enveloped cluster. A k-Means approximator iteratively assigns data points to a cluster of the set of clusters and updates a set of cluster centroids corresponding to the set of clusters. The k-Means approximator assigns data points based at least in part on the set of cluster closures.

Approximate K-means via cluster closures

Image descriptor identifiers are used for content-based search. A plurality of descriptors is determined for an image. The descriptors represent the content of the image at respective interest points identified in the image. The descriptors are mapped to respective descriptor identifiers. The image can thus be represented as a set of descriptor identifiers. A search is performed on an index using the descriptor identifiers as search elements. A method for efficiently searching the inverted index is also provided. Candidate images that include at least a predetermined number of descriptor identifiers that match those of the image are identified. The candidate images are ranked and at least a portion thereof are presented as content-based search results.

Content-based image search

The present invention uses invisible junctions which are a set of local features unique to every page of the electronic document to match the captured image to a part of an electronic document. The present invention includes: an image capture device, a feature extraction and recognition system and database. When an electronic document is printed, the feature extraction and recognition system captures an image of the document page. The features in the captured image are then extracted, indexed and stored in the database. Given a query image, the features in the query image are extracted and compared against those stored in the database to identify the query image. The feature extraction and recognition system of the present invention is integrated into a multifunction peripheral. This allows the feature extraction and recognition system to be used in conjunction with other modules to provide security and annotation applications.

Invisible junction feature recognition for document security or annotation

Methods and computer-storage media having computer-executable instructions embodied thereon that facilitate refining query results using visual cues are provided. Query results are determined in response to an indication of a user query. One or more groups of query results are generated from the query results based on categories of query results that share similar features. Visual cues are associated with each of the query result groups. Visual cues, in association with query result groups, are presented to a user. Query results associated with a selected visual cue may be presented to a user. A refined user query may be generated based on a selected visual cue.

Visual-cue refinement of user query results

Product images are used in conjunction with textual descriptions to improve classifications of product offerings. By combining cues from both text and image descriptions associated with products, implementations enhance both the precision and recall of product description classifications within the context of web-based commerce search. Several implementations are directed to improving those areas where text-only approaches are most unreliable. For example, several implementations use image signals to complement text classifiers and improve overall product classification in situations where brief textual product descriptions use vocabulary that overlaps with multiple diverse categories. Other implementations are directed to using text and images 'training sets' to improve automated classifiers including text-only classifiers. Certain implementations are also directed to learning a number of three-way image classifiers focused only on 'confusing categories' of the text signals to improve upon those specific areas where text-only classification is weakest.

Accurate text classification through selective use of image data

A data partitioning plan is automatically generated that&#x2014;given a data-parallel program and a large input dataset, and without having to first run the program on the input dataset&#x2014;substantially optimizes performance of the distributed execution system that explicitly measures and infers various properties of both data and computation to perform cost estimation and optimization. Estimation may comprise inferring the cost of a candidate data partitioning plan, and optimization may comprise generating an optimal partitioning plan based on the estimated costs of computation and input/output.

Optimizing data partitioning for data-parallel computing

Image search results are obtained by providing weights to visual features to emphasize features corresponding to objects of interest while simultaneously deemphasizing irrelevant or inconsistent features that lead to poor search results. In order to minimize the impact of visual features that are unreliable or irrelevant with respect to the objects of interest in the image, context-dependent weights are provided to detect visual features such that those visual features pertaining to the objects of interest are more heavily weighted than those visual features that pertain to irrelevant or unreliable portions of the image. Visual features may be weighted for images in a searchable database. Training data may be obtained and used in weighting visual features in a query image and, alternatively, in searchable database images.

Image retrieval using discriminative visual features

Text in web pages or other text documents may be classified based on the images or other objects within the webpage. A system for identifying and classifying text related to an object may identify one or more web pages containing the image or similar images, determine topics from the text of the document, and develop a set of training phrases for a classifier. The classifier may be trained and then used to analyze the text in the documents. The training set may include both positive examples and negative examples of text taken from the set of documents. A positive example may include captions or other elements directly associated with the object, while negative examples may include text taken from the documents, but from a large distance from the object. In some cases, the system may iterate on the classification process to refine the results.

Matching text to images

Growing community of inventors

Cupertino, CA, United States of America

Qifa Ke