Microsoft Technology Licensing, LLC

Gabriella Kazai

Pushmeet Kohli

Natasa Milic-Frayling

John Guiver

Milad Shokouhi

Matteo Venanzi

Gavin Smyth

Gerard Oleksik

Eduarda Mendes Rodrigues

Jamie Costello

Crowdsourcing systems with machine learning are described. Specifically, item-label inference methods and systems are presented, for example, to provide aggregated answers to a crowdsourced task in a manner achieving good accuracy even where observed data about past behavior of crowd members is sparse. In various examples, an item-label inference system infers variables describing characteristics of both individual crowd workers and communities of the workers. In various examples, an item-label inference system provides aggregated labels while considering the inferred worker characteristics and the inferred characteristics of the worker communities. In examples the item-label inference system provides uncertainty information associated with the inference results for selecting workers and generating future tasks.

Crowdsourcing system with community learning

Crowdsourcing systems with machine learning are described, for example, to aggregate answers to a crowdsourced task in a manner achieving good accuracy even where observed data about past behavior of crowd members is sparse. In various examples a machine learning system jointly learns variables describing characteristics of both individual crowd workers and communities of the workers. In various examples, the machine learning system learns aggregated labels. In examples learnt variables describing characteristics of an individual crowd worker are related, by addition of noise, to learnt variables describing characteristics of a community of which the individual is a member. In examples the crowdsourcing system uses the learnt variables describing characteristics of individual workers and of communities of workers for any one or more of: active learning, targeted training of workers, targeted issuance of tasks, calculating and issuing rewards.

Methods of dynamic control of an electronic message system are described. In an embodiment, a system which is separate from a messaging service within the electronic message system generates an event signal which relates to an event external to the messaging service. The event signal is received by the messaging service and this triggers a dynamic update in one or more electronic messages which are identified based on the event signal received. For example, the dynamic update may result in the message content being displayed in a different manner or the message being delivered or deleted. Examples of external events include a community response to a particular message, based on a subset of information about the message which has been shared, the location of one or more users and a change in membership of a group.

Dynamic control of an electronic message system

Tagging of resources in order to associate them is described. In an embodiment it is possible to assign tags to resources or switch between resources with different tags, in the context of current work being undertaken by a user. For example, from a single application window that is currently used, a user is able to switch to other resources by tag in an embodiment. Different embodiments illustrate how tags may have multiple purposes and those purposes may evolve thus enabling associated resources to be exposed in different ways. In some embodiments rich metadata about tag usage and/or the use of resource associated with a tag is stored and used to provide additional functionality. Examples are described in which persistence of resources associated with tags is provided and may involve representation of resources such as by duplication of files or by creating alternative representations of resources.

Growing community of inventors

Bishops Stortford, United Kingdom

Gabriella Kazai