Netflix Inc.

Christos Kalantzis

Ioannis Papapanagiotou

Minh Do

Shailesh Birari

Jason Cacciatore

Shyam Singh

In one embodiment, a data reconciliation engine works with data store nodes included in a distributed data store system to ensure consistency between the data store nodes. In operation, the data reconciliation receives a different data snapshot from each of the data store nodes. In response, the data reconciliation engine generates one or more recommendations designed to resolve inconsistencies between the data snapshots. The data reconciliation engine then transmits each recommendation to a different data store node. Because the data reconciliation engine performs many of the resource-intensive operations included in the data reconciliation process, the resources of the data store nodes may focus primarily on processing client requests instead of performing data reconciliation operations. Consequently, unlike conventional data store node based reconciliation applications, the data reconciliation engine may process large volumes of data without unacceptably increasing the time required for the distributed data store system to respond to client requests.

Techniques for performing data reconciliation in distributed data store systems

In various embodiments, a node manager configures a 'new' node as a replacement for an 'unavailable' node that was previously included in a distributed data store. First, the node manager identifies a source node that stores client data that was also stored in the unavailable node. Subsequently, the node manager configures the new node to operate as a slave of the source node and streams the client data from the source node to the new node. Finally, the node manager configures the new node to operate as one of multiple masters nodes in the distributed data store. Advantageously, by configuring the node to implement a hybrid of a master-slave replication scheme and a master-master replication scheme, the node manager enables the distributed data store to process client requests without interruption while automatically restoring the previous level of redundancy provided by the distributed data store.

Techniques for warming up a node in a distributed data store

In an embodiment, a method comprises receiving an input defining organization of one or more universal data elements; generating a universal schema that defines organization of the one or more universal data elements based on the input; associating the universal schema with a specialized plug-in, wherein the specialized plug-in maps the one or more universal data elements with data elements in one or more specialized databases maintained by a specialized data store; receiving, from a computer, a request to receive data stored in the specialized data store and comprising parameters based on the universal schema; transforming the request using the specialized plug-in to produce a specialized query; sending the specialized query to the specialized data store to execute the specialized query; receiving a specialized result set from the specialized data store; sending, to the computer, a result set based on specialized result set.

Growing community of inventors

San Jose, CA, United States of America

Christos Kalantzis