Microsoft Technology Licensing, LLC

Aspen Technology, Inc.

John Guiver

Matteo Venanzi

John Michael Winn

Dmitriy Meyerzon

Alexander Armin Spengler

Ivan Korostelev

Nikita Voronkov

Vladimir V Gvozdev

Pushmeet Kohli

Pavel Myshkov

Yordan Kirilov Zaykov

Elena Pochernina

Paul Turner

Brian Lines

S Steven Treiber

Omar Zia Khan

Michael James Taylor

James Edelen

Milad Shokouhi

Hui Li

Thomas Minka

Jeffrey L Wight

Gabriella Kazai

Martin Kukla

Jan-Ove Almli Karlberg

Åge Andre Kvalnes

Helge Grenager Solheim

Natasa Milic-Frayling

Nicholas Eric Craswell

Hong Zhao

Onno R Zoeter

Samuel Alexander Webster

Stephen Robertson

Elena Zheleva

Anthony Arnold Wieser

Martin Szummer

Junyi Chai

Andrei-Alin Corodescu

Simon Thoresen Hult

Marius Bunescu

Eduarda Mendes Rodrigues

Chia-Jiun Tan

Thuy Thanh Duong

Andrei Razvan Popov

Edward Lloyd Snelson

Guillermo Sentoni

Noura Farra

Omar Faruk

Hadi Abbass Kotaich

Sravya Narala

Jian Li

John Fitzgerald Bronskill

Sanil Rajput

Kashif Dastgir

Saravanakumar Rajmohan

Artyom Sadovsky

In various examples there is a computer-implemented method of database construction. The method comprises storing a knowledge graph comprising nodes connected by edges, each node representing a topic. Accessing a topic type hierarchy comprising a plurality of types of topics, the topic type hierarchy having been computed from a corpus of text documents. One or more text documents are accessed and the method involves labelling a plurality of the nodes with one or more labels, each label denoting a topic type from the topic type hierarchy, by, using a deep language model; or for an individual one of the nodes representing a given topic, searching the accessed text documents for matches to at least one template, the template being a sequence of words and containing the given topic and a placeholder for a topic type; and storing the knowledge graph comprising the plurality of labelled nodes.

Knowledge base with type discovery

Examples described herein generally relate to a computer system for generating a knowledge graph storing a plurality of entities and to displaying a topic page for an entity in the knowledge graph. The computer system performs a mining of source documents within an enterprise intranet to determine a plurality of entity names. The computer system generates an entity record within the knowledge graph for a mined entity name based on an entity schema and the source documents. The entity record includes attributes aggregated from the source documents. The computer system receives a curation action on the entity record from a first user. The computer system updates the entity record based on the curation action. The computer system displays an entity page including at least a portion of the attributes to a second user based on permissions of the second user to view the source documents.

Enterprise knowledge graph

Examples described herein generally relate to a computer system including a knowledge graph storing a plurality of entities. A mining of a set of enterprise source documents within an enterprise intranet is performed, by a plurality of knowledge mining toolkits, to determine a plurality of entity names. The plurality of entity names are linked based on entity metadata by traversing various relationships between people, files, sites, groups, associated with entities. An entity record is generated within a knowledge graph for a mined entity name from the linked entity names based on an entity schema and ones of the set of enterprise source documents associated with the mined entity name. The entity record includes attributes aggregated from the ones of the set of enterprise source documents associated with the mined entity name.

Enterprise knowledge graphs using multiple toolkits

Mining of a set of enterprise source documents within an enterprise intranet is performed, by a plurality of knowledge mining toolkits, to determine a plurality of entity names. A plurality of entity records are generated within a knowledge graph for mined entity names from the entity names based on an entity schema and ones of the set of enterprise source documents associated with the mined entity names. Pattern recognition is applied to an active document using an enterprise named entity recognition (ENER) system to identify potential entity names within the document that match a respective one of a plurality of entity records in the knowledge graph. One or more matching entity names are annotated within the document with information from the knowledge graph for the respective ones of the plurality of entity records. The annotated information is displayed with the active document.

Annotations for enterprise knowledge graphs using multiple toolkits

Examples described herein generally relate to a computer system including a knowledge graph storing a plurality of entities. The computer system generates an Aho Corasick trie including an entity name for each of the plurality of entities in the knowledge graph. The computer system compares a document viewed by a user to a plurality of templates defining potential entity names to identify extracts of the document matching at least one of the plurality of templates. The computer system applies the document to the Aho Corasick trie to determine potential entity names within the document that each match a respective one of the plurality of entities in the knowledge graph. The computer system annotates one or more matching entity names within the document with information from the knowledge graph for the respective ones of the plurality of entities to show, for example, a topic card providing information about the respective entities.

Document annotation based on enterprise knowledge graph

Examples described herein generally relate to a computer system including a knowledge graph storing a plurality of entities. The computer system compares source documents within an enterprise intranet to a plurality of templates defining potential entity attributes to identify extracts matching at least one of the plurality of templates. The computer system parses the extracts according to respective templates of the plurality of templates that match the extracts to determine instances. The computer system performs incremental clustering on a number of the instances to determine potential entity names. The computer system queries the knowledge graph with the potential entity names to obtain a set of candidate entity records. The computer system links the potential entity names with at least partial matching ones of the set of candidate entity records to define updated matching candidate entity records. The computer system updates the knowledge graph with the updated matching candidate entity records.

Incremental clustering for enterprise knowledge graph

Crowdsourcing using active learning is described, for example, to select pairs of tasks and groups of workers so that information gained about answers to the tasks in the pool is optimized. In various examples a machine learning system learns variables describing characteristics of communities of workers, characteristics of workers, task variables and uncertainty of these variables. In various examples, the machine learning system predicts task variables and uncertainty of the predicted task variables for possible combinations of communities of workers and tasks. In examples the predicted variables and uncertainty are used to calculate expected information gain of the possible combinations and to rank the possible combinations. In examples, the crowdsourcing system uses the expected information gain to allocate tasks to worker communities and observe the results; the results may then be used to update the machine learning system.

Adaptive task assignment

Crowdsourcing systems with machine learning are described. Specifically, item-label inference methods and systems are presented, for example, to provide aggregated answers to a crowdsourced task in a manner achieving good accuracy even where observed data about past behavior of crowd members is sparse. In various examples, an item-label inference system infers variables describing characteristics of both individual crowd workers and communities of the workers. In various examples, an item-label inference system provides aggregated labels while considering the inferred worker characteristics and the inferred characteristics of the worker communities. In examples the item-label inference system provides uncertainty information associated with the inference results for selecting workers and generating future tasks.

Crowdsourcing system with community learning

Generating and utilizing personalized predictive models are provided. When an electronic input is received, a generic predictive model is used to predict a user response to the input. After a prescribed period of time, an analysis is performed to determine the user's actual response to the input, as well as, the user's actual responses to other inputs of the same type. Training is performed on the generic predictive model to generate a new and personalized predictive model based on the user's actual responses to the analyzed inputs. The personalized predictive model is then utilized for predicting user response to future inputs of the same type. At a prescribed frequency, the generated personalized predictive model is updated by analyzing actual user responses to predictions provided by the personalized predictive model.

Personalized predictive models

Crowdsourcing systems with machine learning are described, for example, to aggregate answers to a crowdsourced task in a manner achieving good accuracy even where observed data about past behavior of crowd members is sparse. In various examples a machine learning system jointly learns variables describing characteristics of both individual crowd workers and communities of the workers. In various examples, the machine learning system learns aggregated labels. In examples learnt variables describing characteristics of an individual crowd worker are related, by addition of noise, to learnt variables describing characteristics of a community of which the individual is a member. In examples the crowdsourcing system uses the learnt variables describing characteristics of individual workers and of communities of workers for any one or more of: active learning, targeted training of workers, targeted issuance of tasks, calculating and issuing rewards.

Technologies are generally provided far a hierarchical, feature teed statistical model that cm be used for personalized classification or predictions within a community of users. Personalization refers to learning about the habits and characteristics of individual users and adapting user experiences based on that learning. The model may be used in a communication application to predict user actions on incoming email messages and to help users triage email by making personalized suggestions based on the model predictions. A community of users associated together with the communication application may be incorporated together into a single model to enable for continuous fine-grain interaction between intelligence learned from the community of users as a whole and that learned from individual users. The single model may allow a seamless progression between predictions for a completely new user based on community observations and highly personalized predictions for a long-term user based on individual observations.

Hierarchical statistical model for behavior prediction and classification

Recommending a media item may include, for example, a statistical model of media consumption is applied to media session consumption data from a community of users to infer parameters of the model. The model comprises a first probability distribution for each user defining a likelihood of the user having a latent characteristic for a session, and a second probability distribution for each latent characteristic defining a likelihood of a user selecting a media item given the latent characteristic. In another example, the inferred parameters are provided to a recommendation engine arranged to recommend media items. The recommendation engine uses the model with inferred parameters and data describing media items newly consumed by a user to infer a current latent characteristic for a current session of the user, and uses them to generate recommended media items for the user in the current session based on the current latent characteristic.

Media item recommendation

A constrained non-linear approximator for empirical process control is disclosed. The approximator constrains the behavior of the derivative of a subject empirical model without adversely affecting the ability of the model to represent generic non-linear relationships. There are three stages to developing the constrained non-linear approximator. The first stage is the specification of the general shape of the gain trajectory or base non-linear function which is specified graphically, algebraically or generically and is used as the basis for transfer functions used in the second stage. The second stage of the invention is the interconnection of the transfer functions to allow non-linear approximation. The final stage of the invention is the constrained optimization of the model coefficients such that the general shape of the input/output mappings (and their corresponding derivatives) are conserved.

Computer method and apparatus for constraining a non-linear approximator of an empirical process

An information retrieval system is described for retrieving a list of documents such as web pages or other items from a document index in response to a user query. In an embodiment a prediction engine is used to predict both explicit relevance information such as judgment labels and implicit relevance information such as click data. In an embodiment the predicted relevance information is applied to a stored utility function that describes user satisfaction with a search session. This produces utility scores for proposed lists of documents. Using the utility scores one of the lists of documents is selected. In this way different sources of relevance information are combined into a single information retrieval system in a principled and effective manner which gives improved performance.

Information retrieval system

Methods to enable optimization of discontinuous rank metrics are described. The search scores associated with a number of search objects are written as score distributions and these are converted into rank distributions for each object in an iterative process. Each object is selected in turn and the score distribution of the selected object is compared to the score distributions of each other object in turn to generate a probability that the selected object is ranked in a particular position. For example, with three documents the rank distribution may give a 20% probability that a document is ranked first, a 60% probability that the document is ranked second and a 20% probability that the document is ranked third. In some embodiments, the rank distributions may then be used in the optimization of discontinuous rank metrics.

Optimization of discontinuous rank metrics

Growing community of inventors

Saffron Walden, United Kingdom

John Guiver