Microsoft Technology Licensing, LLC

Franco Salvetti

David D Ahn

Giovanni Lorenzo Thione

Michael Paul Bieniosek

Srinivas Vadrevu

Daniel Marantz

Gianluca Donato

Krishnan Thazhathekalam

Tejas Girish Nadkarni

Andrea Ryerson Burbank

Toby Takeo Sterrett

Ian Collins

Riccardo Turchetto

Christopher Jon Jewell

Andrei Peter Makhanov

Ying Tu

Martin Henk Van Den Berg

Livia Polanyi

Richard Stamford Crouch

Micol Marchetti-Bowick

Grigor Shirakyan

David Simpson

Justin John Trobec

Hamed Esfahani

Methods and systems are provided for improving efficiency and precision in factual question answering in a search environment. In aspects, the methods and systems involve offline identification of answers correlated with one or more factual questions that are generated offline for each document within a document collection. Thereafter, upon receiving a factoid query during runtime, candidate documents that are retrieved in response to the factoid query are scanned for a factual question that matches the factoid query. For each factual question that matches the factoid query, a corresponding candidate answer can be identified. Thereafter, each candidate answer can be assigned a score that is a function of the number of candidate documents in which the candidate answer was identified and the relevancy of those candidate documents to the factoid query. The candidate answer having the highest score can be provided to the user in response to the factoid query.

Ranking for efficient factual question answering

Architecture that enables search engines to meet user expectations for search results (e.g., questions-answers) by improving on result consistency. This is attained by declining to answer queries when it is known by the system that the system is unable to answer more or equivalently prominent queries in the same query class in a vast majority of cases. To achieve consistency, queries are categorized into classes and then the queries of a specific class are segmented into clusters. The answer density in each cluster is then computed to determine the consistency of that class of queries. Clusters with a relatively low answer density are then suppressed to improve consistency for the user.

Search engine results system using entity density

Architecture that enables an optional display of a longer version of each subsnippet in response to user interactions such as clicking, hovering, or other suitable form of interaction. More specifically, options are provided to display additional text from a search result at the point where a subsnippet (a subsegment in a snippet that is delimited by ellipses) ends. Selecting suitable boundaries for both initial subsnippets and expanded subsnippets enables relevant information to be presented and increased readability.

Subsnippet handling in search results

Summaries of entities (e.g., people, places, things, concepts, etc.) may provide additional useful information to user. For example, a search engine may provide a summary of an entity within search results. A category (e.g., 'writer', 'politician', etc.) of the entity that is short and concise may be advantageous to provide within a summary of the entity. The category may allow a user to quickly determine whether the information of the entity relates to the intended entity (e.g., search results of an entity as &#x201c;a writer&#x201d; vs. search results of an entity as &#x201c;a politician&#x201d;). Potential categories and summary text may be extracted from pre-labeled data. The potential categories and summary text may be intersected to determine a set of candidate categories that may be ranked. An entity category having a desired ranked may be determined as the entity category that describes the entity in a desired way.

Entity category extraction for an entity that is the subject of pre-labeled data

Architecture that addresses at least the problem of ranking the relevant attributes for a given entity within the context of a structured knowledge base (SKB). The architecture utilizes the attribute, entity type statistics, and the taxonomy of the attributes to consistently and efficiently rank attributes for each and every type of entity in the SKB. Using the SKB, intermediate features are computed, including the importance or popularity each entity type for every entity, inverse document frequency (IDF) computation for each attribute on a global basis, IDF computation for entity types, and the popularity of attributes for each entity type. The intermediate features are aggregated to obtain a final feature set, which can be used in combination with human judgments to train a machine learned classifier model to produce and predict a relevance score for a given entity and each of its attributes. The attributes are ranked for each entity using this score.

Ranking relevant attributes of entity in structured knowledge base

An ecosystem that enables content providers to decorate search results with interactive content. The searching user can then interact with the content and view the content without leaving the search results page. The content provider sends content and metadata to a content enrichment enabler that transforms the content into an enriched content, and receives back from the enrichment enabler a location identifier which includes information that identifies the provider and the location of the enriched content. The content provider then embeds the identifier in each of the content provider webpages for which such content has been produced. The identifier is indexed by a search engine to identify the interactive content and source thereof for surfacing on a search results page. The search result is decorated with an indicator that the user recognizes as the availability of the enriched content, and uses to access the content via the web page.

Decoration of search results by third-party content providers

Architecture that uses content from off-page data sources such as feeds (e.g., yellow pages, coupons, social networks, commerce, etc.) to present additional, relevant information in association with search results. The additional and relevant information is directly relevant to the implicit task the user is trying to accomplish. The architecture includes online and offline mechanisms that identify an entity represented on a web page and look-up information related to that entity in disparate data sources. Relevance heuristics are employed to determine which of the available entity data to show in the caption given the user query, the web page, and the underlying user task (other known information about the user such as geographic location).

Use of off-page content to enhance captions with additional relevant information

Architecture that inserts one or more label items in search result entries. In addition to the typical search result caption (title, snippet, and link), the architecture includes the label component of one or more of the label items in the result entry. The number and type of label annotations are based on the query. When a particular label item is selected (e.g., hover, mouse click), a presentation component (e.g., expansion object, pop-up window) launches proximate to a label item in response to interaction with the label item and presents additional information from the target webpage. The additional information can include an action and data related to the search result entry and the target webpage. The data can be obtained from a data source other than the target webpage.

Automatic information presentation of data and actions in search results

The present technology is related to identifying, from within a corpus of documents, a subject (e.g., person, location, date, etc.) that is relevant to a topic and that is usable to enhance a topic-describing document. Documents within the corpus of documents share a link structure, such that some documents include hyperlinks that enable navigation to the topic-describing document, and the topic-describing document includes hyperlinks that enable navigation to other documents. Text of documents within the corpus is parsed to identify the subject, and a context of the subject suggests a degree of relevance of the subject to the topic. An enhancement type of the subject is determined, and a version of the topic-describing document is enhanced to include a presentation of the subject.

Enhancing a document with supplemental information from another document

Technologies are described herein for coreference resolution in an ambiguity-sensitive natural language processing system. Techniques for integrating reference resolution functionality into a natural language processing system can processes documents to be indexed within an information search and retrieval system. Ambiguity awareness features, as well as ambiguity resolution functionality, can operate in coordination with coreference resolution. Annotation of coreference entities, as well as ambiguous interpretations, can be supported by in-line markup within text content or by external entity maps. Information expressed within documents can be formally organized in terms of facts, or relationships between entities in the text. Expansion can support applying multiple aliases, or ambiguities, to an entity being indexed so that all of the possibly references or interpretations for that entity are captured into the index. Alternative stored descriptions can support retrieval of a fact by either the original description or a coreferential description.

Coreference resolution in an ambiguity-sensitive natural language processing system

Consumers receive module-computed composite reviews that are lively, informative, coherent, and representative of a larger underlying collection of reviews. Representative phrases from reviews are extracted and aggregated into coherent sentences to create the composite review. Clear automatable criteria are provided to define coherence and other qualities, such as representativeness, liveliness, and informativity. Sentence coherence criteria involve syntax, shared vocabulary, phrase connectors, and phrase sentiment polarity, for instance. Phrase representativeness criteria involve review ratings and derived phrase ratings, for instance. Phrase liveliness criteria involve sentiment expression frequency, superlatives, comparatives, degree modifiers, affect activation scores, and affect imagery scores, for instance. Phrase informativity criteria involve product-specific words, review length, and recency, for instance. Prohibited language is filtered out. Composite reviews are automatically distributed, e.g., in response to a web search on the reviewed product. Reviews can be generated with a repeatability and rapidity not attainable by human performance alone.

Automatic generation of digital composite product reviews

Answers are provided to users in response to queries as a supplement to any responsive documents. Query formats for entity and attribute combinations are identified. The query formats can be substituted with entity and attribute combinations that have a corresponding attribute value to form a list of answered queries. The attribute value corresponding to an answered query can be provided when a query is received that matches an answered query.

Query pattern generation for answers coverage expansion

Concept disambiguation is provided for search queries by analyzing search results in conjunction with an ontology of concepts. An ontology of concepts is identified, and at least one document is associated with each concept. The document associated with a concept is representative of the concept and used to generate a concept signature. When a search query is received, it is processed to obtain search results. The search results are used to generate a search results signature, which is compared to the concept signatures to identify one or more concepts that are relevant to the search query.

Concept disambiguation via search engine search results

Referring expressions are identified for concepts by analyzing search query and result selection information. An ontology of concepts is identified, and at least one document is associated with each concept. The document associated with a concept is representative of the concept. Search query information from a search engine is analyzed to identify search queries that resulted in user selections of documents associated with the concepts. Referring expressions that refer to the concepts are identified based on the search queries that resulted in user selections of documents corresponding with the concepts. After identifying referring expressions for concepts, search queries may be mapped to referring expressions to identify concepts to which the search queries pertain, and search result pages may be generated based on knowledge of the concepts.

Growing community of inventors

San Francisco, CA, United States of America

Franco Salvetti