Sun Microsystems, Inc.

Google Inc.

Oracle America, Inc.

Grzegorz Jan Czajkowski

James C Dehnert

Matthew Harold Austern

Aart J C Bik

Grzegorz Malewicz

Laurent Daynes

Laurent P Daynes

Laurent Philippe Daynès

Glenn Carlton Skinner

Stephen Charles Hahn

Krzysztof Palacz

Naty Leiser

Ciaran J Bryce

Peter James Soper

Joseph S Beda, Iii

Ilan Horn

Sunil Soman

Mohamed Elbassiony Mohamed Abou El Alaa Khalefa

Michal Wegiel

Andrew McClure

Mario I Wolczko

Yonggang Zhao

Michael J Jordan

Rodrick Ison Evans

Jerry Zhao

Laurent Daynès

Charles Reiss

Nathaniel J Nystrom

Chrislain Razafimahefa

Aart Jc Bik

Data are maintained in a distributed computing system that describe a graph. The graph represents relationships among items. The graph has a plurality of vertices that represent the items and a plurality of edges connecting the plurality of vertices. At least one vertex of the plurality of vertices includes a set of label values indicating the at least one vertex's strength of association with a label from a set of labels. The set of labels describe possible characteristics of an item represented by the at least one vertex. At least one edge of the plurality of edges includes a set of label weights for influencing label values that traverse the at least one edge. A label propagation algorithm is executed for a plurality of the vertices in the graph in parallel for a series of synchronized iterations to propagate labels through the graph.

Label propagation in a distributed system

Data are maintained in a distributed computing system that describe a directed graph representing relationships among a set of items. The directed graph models a condition having an associated problem. The directed graph has graph components having associated data fields. The relationships are analyzed to identify a solution to the problem. As part of the analysis, a new value for the data field associated with a graph component is identified responsive to an operation performed during the analysis. The new value is compared with an existing value of the data field, and the data field is modified. The modification may include inserting the new value into an overflow vector of data, and replacing the existing value in the data field with exception information identifying the location of the new value. An exception flag associated with the data field is set to indicate that the exception information is being used.

Data storage in a graph processing system

Data are received at a worker system in a distributed computing system that describe a graph representing relationships among a set of items. The graph models a condition having an associated problem. The graph has graph components having associated data fields. The received data are stored in a backup table, and the relationships are analyzed to identify a solution to the problem. As part of the analysis, a new value for the data field associated with a graph component is identified and compared with an existing value of the data field, and the data field is modified. The modified data field is stored in a delta table representing a change to the backup table.

Data backup in a graph processing system

Data are maintained in a distributed computing system that describe a directed graph representing relationships among items. The directed graph has a plurality of vertices representing the items and has edges with values representing distances between the items connected by the vertices. A multiple reference point algorithm is executed for a plurality of the vertices in the directed graph in parallel for a series of synchronized iterations to determine shortest distances between the vertices and the source vertex. After executing the algorithm on the vertices, value pairs associated with the vertices are aggregated. The aggregated value pairs indicate shortest distances from the respective vertices to the source vertex. The aggregated value pairs are outputted.

Multiple reference point shortest path algorithm

A value is distributed in a distributed computing system having a master system in communication with a plurality of worker systems. Partitions of a graph are assigned to the worker systems. The graph represents relationships among a set of tangible items that model a real-world condition having an associated problem. Configuration information is determined that describes a configuration of the distributed computing system. A distribution scheme is selected for distributing a value from the master system to the plurality of worker systems based on the configuration information. The value is distributed from the master system to the worker systems according to the selected distribution scheme. The worker systems are configured to use the value to produce an output representing a solution to the real-world problem.

Distributing global values in a graph processing system

Executing a confined recovery in a distributed system having a plurality of worker systems including a failed worker system at a current superstep. The confined recovery includes determining states of the partitions of the worker systems during the supersteps preceding the current superstep, and determining a recovery initiation superstep preceding the current superstep in which all messages for recovery initiation superstep are available. The recovery initiation superstep is determined responsive to determining the states of the partitions. Additionally, a recovery set of partitions is determined for which messages in supersteps after the recovery initiation superstep are not available. The worker systems having the partitions in the recovery set are instructed to execute the defined function for the partitions in the recovery set starting at the recovery initiation superstep to recover the lost exchanged messages.

Confined recovery in a distributed computing system

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for clustering for parallel processing. One of the methods includes providing virtual machines with an interface to a shuffle service, the shuffle service executing external of the virtual machines. The method includes receiving data records through the interface, each data record having a key and a value. The method includes partitioning the data records, using the shuffle service, according to the respective keys. The method includes providing a part of the partitioned data records through the interface to the virtual machines, wherein data records having the same key are provided to the same virtual machine. Each of the virtual machines can execute on a host machine and each of the virtual machine is a hardware virtualization of a machine.

Clustering for parallel processing

Growing community of inventors

Mountain View, CA, United States of America

Grzegorz Jan Czajkowski