Microsoft Technology Licensing, LLC

Antony Rowstron

Paolo Costa

Thomas Karagiannis

Gregory O'Shea

Austin N Donnelly

Hitesh Ballani

Dushyanth Narayanan

Manuel Costa

Miguel Castro

Thomas Christian Zahn

Jon Crowcroft

Christos Gkantsidis

Miguel Oom Temudo De Castro

Christopher Wilson

Peter Bernard Key

Manuel Silverio Da Silva Costa

Sameh Elnikety

Eno Thereska

Dimitrios Vytiniotis

Orion Tamlin Hodson

Richard Mortier

Virajith Jalaparti

An application programming interface (API) supporting server and key based networking is described. In an embodiment, the API receives either a key or a server address from a service running on a server in a direct-connect topology and returns data which identifies suitable next hops for transmission of a packet of data which has a destination of the received server address or of a server address which is encoded within the received key. In another embodiment, the key also encodes information specifying alternative server addresses for use in the event that the original server is unreachable. This information may also be used to define servers for replication of the key. A further embodiment describes a method of queuing packets for transmission against multiple links, where the packet is transmitted on the first available link and at this time is removed from the queues for the other links.

API supporting server and key based networking

Methods of generating filters automatically from data processing jobs are described. In an embodiment, these filters are automatically generated from a compiled version of the data processing job using static analysis which is applied to a high-level representation of the job. The executable filter is arranged to suppress rows and/or columns within the data to which the job is applied and which do not affect the output of the job. The filters are generated by a filter generator and then stored and applied dynamically at a filtering proxy that may be co-located with the storage node that holds the data. In another embodiment, the filtered data may be cached close to a compute node which runs the job and data may be provided to the compute node from the local cache rather than from the filtering proxy.

Generation and application of correctness-enforced executable filters

Methods of predicting datacenter performance to improve provisioning are described. In an embodiment, a resource manager element receives a request from a tenant which describes an application that the tenant wants executed by a multi-resource, multi-tenant datacenter. The request that has been received is mapped to a set of different candidate resource combinations within the datacenter, where each candidate resource combination can be used to execute the application in a manner which satisfies a high level constraint specified within the request. This mapping may, for example, be performed using a combination of benchmarking and an analytical model. In some examples, each resource combination may comprise a number of virtual machines and a bandwidth between those machines. Data relating to at least a subset (and in some examples, two or more) of the candidate resource combinations is then presented to the tenant.

Predicting datacenter performance to improve provisioning

Methods of offering network performance guarantees in multi-tenant datacenters are described. In an embodiment, a request for resources received at a datacenter from a tenant comprises a number of virtual machines and a performance requirement, such as a bandwidth requirement, specified by the tenant. A network manager within the datacenter maps the request onto the datacenter topology and allocates virtual machines within the datacenter based on the available slots for virtual machines within the topology and such that the performance requirement is satisfied. Following allocation, stored residual capacity values for elements within the topology are updated according to the new allocation and this updated stored data is used in mapping subsequent requests onto the datacenter. The allocated virtual machines form part of a virtual network within the datacenter which is allocated in response to the request and two virtual network abstractions are described: virtual clusters and virtual oversubscribed clusters.

Offering network performance guarantees in multi-tenant datacenters

A deadline-aware network protocol is described. In an example, data transfer at a transport layer entity of a packet-based communication network is controlled by receiving a request for network resources for a data flow from a network element and allocating network resources to the data flow. The data flow comprises a number of data packets associated with an application, and the request comprises a factor relating to a time deadline associated with the application. The network resources allocated depend on the factor relating to the time deadline. In examples, the network resource can be a bandwidth or data rate allocated to the data flow, and the factor can be a data rate sufficient to complete the data flow within the time deadline. In examples, the network resources are allocated greedily, such that requests are fully satisfied whenever possible, and the network resources are fully utilized.

Deadline-aware network protocol

Resource optimization for online services is described. In one example, objects (such as mailboxes or other data associated with an online service) are assigned to network elements (such as servers) by inferring a relationship graph from log data relating to usage of the online service. The graph has a node for each object, and connections between each pair of objects having data items in common. Each connection has a weight relating to the number of common data items. The graph is partitioned into a set of clusters, such that each cluster has nodes joined by connections with a high weight relative to the weight of connections between nodes in different clusters. The objects are then distributed to the network elements such that objects corresponding to nodes in the same cluster are located on the same network element.

Resource optimization for online services

Content transfer between nodes in a communications network is required in many situations. For example, between fixed nodes in a data center, between a fixed node such as a base station and a mobile node such as a mobile telephone and between mobile nodes themselves such as personal digital assistants. In an embodiment a content transfer process for use between nodes where one node knows the size of content to be transferred is described. For example, a file is divided into blocks to be sent between the nodes. A node receiving a block keeps a record of the received blocks and sends that record to the sending node at certain times. By controlling the times at which the records are sent the number of messages that need to be sent to transfer the records is low and the blocks are transferred in an efficient manner.

Content transfer

Supporting distributed key-based processes is described. In an embodiment, servers at a data center provide a key-based process for carrying out computationally expensive tasks and are connected using point to point connections in a geometric topology such as a torus. In an example, aggregation trees are built on top of the geometric topology of the data center, each aggregation tree being a sequence of servers in the data center that forms a tree structure. In an embodiment packets of data are sent from the leaves of the trees to the root and at each server along the tree the packets are aggregated using a combiner function of the key-based process. In an embodiment, if a server fails, the trees are dynamically recomputed and a recovery phase is triggered to resend any packets lost at the failed server. In some embodiments, packets are scheduled by inspecting the content of the packets.

Supporting distributed key-based processes

Content discovery and content transfer between mobile communications nodes is often required, for example, to synchronize maps, traffic hot spot information and the like for vehicle mounted satellite navigation devices, pedestrian hand-held personal digital assistants and other mobile communications nodes. In an embodiment, content discovery processes using a tree-based data structure for transferring data between two mobile communication nodes is described.

Content discovery and transfer between mobile communications nodes

Methods of querying a large number of endsystems are described in which metadata is replicated between endsystems. When a query is injected, an available endsystem receives a message relating to the query which identifies a range of endsystems for which that available endsystem is responsible. The available endsystem then generates completeness data for the range of endsystems based on data stored at the endsystem and this completeness data is transmitted to the sender of the message. The methods may be implemented using device-executable instructions which may be stored on device readable media.

Infrastructure to disseminate queries and provide query results

Content discovery and content transfer between mobile communications nodes is often required, for example, to synchronize maps, traffic hot spot information and the like for vehicle mounted satellite navigation devices, pedestrian hand-held personal digital assistants and other mobile communications nodes. In an embodiment, content discovery processes using data structures designed in particular ways and sent using a single packet enable nodes to identify files to be transferred whilst keeping overheads down. In an embodiment a first and a second data structure are used, each being formed from slightly different information about a subset of the files to be synchronized. In another embodiment a single tree-based data structure is used. A process for transferring data between two mobile communications nodes is also described.

Improved storage systems which use write off-loading are described. When a request to store some data in a particular storage location is received, if the particular storage location is unavailable, the data is stored in an alternative location. In an embodiment, the particular storage location may be unavailable because it is powered down or because it is overloaded. The data stored in the alternative location may be subsequently recovered and written to the particular storage location once it becomes available.

Growing community of inventors

Cambridge, United Kingdom

Antony Rowstron