Google Inc.

Apple Inc.

Yahoo! Inc.

Alexander Shraer

Arif Abdulhusein Merchant

Brian Frank Cooper

Artyom Sharov

Atul Adya

Amin Vahdat

Junlan Zhou

Kfir Lev-Ari

Vishesh Khemani

Hong Liu

Ryohei Urata

Christopher Lyle Johnson

Ori Herrnstadt

Yannick L Sierra

Marcus Felipe Fontoura

Vanja Josifovski

Maxim Gurevich

Raghav V Pisolkar

Monia Ghobadi

Nicolas A Favre-Felix

Nathan L Williams

Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, a leader is selected from among candidate computing clusters. To make this selection, an activity monitor predicts or monitors the workload of one or more clients. Different activities of the workload are given corresponding weights. The delay in performing requested activities, modified by these weights is found, and the candidate leader with the lowest weighted delay is selected as the leader.

Distributed database configuration

Methods, systems, and apparatus for automatic sharding and load balancing in a distributed data processing system. In one aspect, a method includes determining workload distribution for an application across worker computers and in response to determining a load balancing operation is required: selecting a first worker computer having a highest load measure relative to respective load measure of the other work computers; determining one or more move operations for a partition of data assigned to the first worker computer and a weight for each move operation; and selecting the move operation with a highest weight the selected move operation.

Weighted auto-sharing

Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, a leader is selected from among candidate computing dusters. To make this selection, an activity monitor predicts or monitors the workload of one or more clients. Different activities of the workload are given corresponding weights. The delay in performing requested activities, modified by these weights is found, and the candidate leader with the lowest weighted delay is selected as the leader.

This application relates to load balancing for a number of partitions of a network-based storage service. Each partition includes a number of server devices and/or network devices located in a data center and configured to provide access to storage resources hosted within the data center. User accounts are assigned to a particular partition such that requests related to a particular user account are routed to that partition. Periodically, a load balancing algorithm is executed to re-assign user accounts to different partitions to rebalance resource consumption across the different partitions. The load balancing algorithm can balance resource consumption for any number of resource types by generating a vector of resource utilization parameters for each user account, sorting the plurality of user accounts into clusters based on the vectors, and mapping at least some user accounts to different partitions.

Rebalancing of user accounts among partitions of a storage service

Weighted auto-sharding

Data items such as files or database records associated with particular applications (such as messaging applications and other applications) can be stored in one or more remote locations, such as a cloud storage system, and synchronized with other devices. The remote storage can be configured such that each application executing on a client device can only view data items stored at the remote location to which the application has permission to access. An access manager on each client device enforces application specific access policies. Storage at the remote location can be secured for each application associated with a user or user account, for example, using isolated containers. The cloud storage of data can be anonymized and anonymous group data can be stored in the cloud storage.

Cloud messaging system

Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, the role of voting is assigned to a number of computing clusters. To make this selection, a task assigning service finds a candidate leader and M computing clusters to be used as voters. This selection may be based on finding latencies between the M computing clusters and the leader.

Distributed storage system with replica selection

Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, an indication that a number N of clusters are to be assigned a replica role and a second number M of the replica-role assigned clusters are to be assigned to a voting role. N computing clusters are selected using workload data, and M of the clusters are assigned to a voting role.

Distributed storage system with replica location selection

Systems and methods for increasing bandwidth in a computer network are provided. A computer network can include a first lower level switch, first and second upper level switches, and first and second passive optical splitters, and a mirror. The first passive optical splitter can have a first port directly coupled to the first upper level switch, a second port directly coupled to the second upper level switch. The second passive optical splitter can have a port directly coupled to a port of the first passive optical splitter, and a port directly coupled to the first lower level switch. The mirror can be coupled to a port of the second passive optical splitter and reflect an optical signal received from the second passive optical splitter to the first upper level switch and second upper level switch through the second passive optical splitter and the first passive optical splitter.

Systems and methods for increasing bandwidth in a computer network

Various methods and apparatuses are provided which may be implemented using one or more computing devices within a networked computing environment to employ publish-subscribe techniques to associate subscriber encoded data files with a set of publisher encoded data files.

Publish-subscribe based methods and apparatuses for associating data files

The present disclosure provides for the determination of bandwidth allocation of inter-block traffic in a data center network. It employs a number of optimization objectives and a heuristic water-filling strategy to avoid producing unnecessary paths and to avoid determining paths that would be unavailable when actually needed. Allocation may be adjusted incrementally upon node and link failure, for instance to perform only the minimal allocation changes necessary. If demand between a source and a destination cannot be satisfied, a decomposition process may be used to allocate remaining demand. One aspect constructs a graph for route computation based on inter-block topology. Here, the graph initially starts with a highest level of abstraction with each node representing a middle block, and gradually reduces the abstraction level to identify paths of mixed abstraction level to satisfy additional demand.

Traffic engineering for large scale data center networks

Aspects and implementations of the present disclosure are directed to a network device storing an address table. Each of the address table entries is assigned to a respective provisioned index value and an instance of a distributed service. The device is configured to add a new service instance to the address table by obtaining an assignable index value for the new service instance, identifying an assigned index value that is a companion index value to the obtained index value, dividing a packet source address set associated with the companion index value into first and second address subsets, and assigning the first address subset to the service instance entry in the address table associated with the assigned companion index value and the second address subset to an entry in the address table associated with the obtained index value and the newly added service instance.

Growing community of inventors

Stanford, CA, United States of America

Alexander Shraer