Yahoo! Inc.

Other

Tvworks, LLC

Venkatachary Srinivasan

Marco Boerries

Brian J Bulkowski

Markus Meyer

Torsten Schulz

Matthias Breuer

Andrew Gooding

Sunil Sayyaparaju

Lie Yang

Zhaowei Charlie Jiang

Min Zhou

Meher Tendjoukian

Hui Guo

Surendra Sadanand Rajam

Raymond Drewry

Rajkumar Iyer

Bhaskar A Parvathaneni

Ashish Shinde

John A Traver

Joseph J Martin

Jonathan I McCormack

Hari Vasudev

Jerald J Singh

Bjørn Ebbesen

Thomas Kraus

David Michael Finnegan

In one aspect, a computerized method for expanding a graphics processing unit (GPU) memory footprint based on a hybrid-memory of a distributed database system (DDBS) includes the step of providing the DDBS. The DDBS is modified to include a plurality of GPUs; providing a local memory of a GPU of the plurality of GPUs. The method includes the step of filling the local memory of the GPU with one or more digests from the DDBS. The method includes the step of running a distributed general-purpose cluster-computing framework instance on the local memory of the GPU. The method includes the step of fetching data from the local memory of the GPU using the distributed general-purpose cluster-computing framework instance. The method includes the step of storing a result of the fetch operation in the DDBS to extend the local memory of the GPU to handle more data than what is fitted into the local memory of the GPU.

Methods and systems for expanding GPU memory footprint based on hybrid-memory

In one aspect, a computerized method for managing tombstones in a node of a Distributed Database Base System (DDBS) includes the step of providing a rule that, for a namespace in a record of the node of the DDBS that allows expiration, mandates that a later generation's expiration time of the namespace never decreases. The computerized method includes the step of determining that an administrator of the DDBS has set an expiration time of the namespace to infinity. The computerized method includes the step of implementing a background process of the DDBS, wherein the background process scans a DDBS node's drive and flags a set of extant tombstones that are no longer covering viable namespaces or viable records. The computerized method includes the step of deleting all the flagged tombstones.

Methods and systems of managing deletes in a database node of a NoSQL database

In one aspect, a computerized method useful for operating a database management system DMBS DBMS in a strong consistency mode comprising includes the step of implementing an atomic transfer of a master node of the DMBS DBMS. The method includes the step of implementing a master node restriction, wherein the master node restriction guarantees no more than one master node is ever available. The information about a set of nodes participating in a cluster of the set node of the DMBS DBMS is maintained to allow a subset of nodes to determine that the one master node is a master of a subset of data stored in the cluster.

Methods and systems of operating a database management system DBMS in a strong consistency mode

A computerized method useful for node cluster formation and management in a distributed database system includes the step of providing a set of database nodes in a database cluster. Each node of the set of database nodes comprises a cluster management module, a heartbeat module, and a globally unique node identifier. The cluster management module manages cluster membership by maintaining a succession list. The succession list comprises a list of all current members in the database cluster. The heartbeat module communicates a pulse message to every other node in the set of database nodes. The method includes the step of maintaining, with each heartbeat module of each database node, an adjacency list.

Methods and systems of database cluster formation and management

In one aspect, a computerized method for managing consistency and availability tradeoffs in a real-time operational database management system (DBMS) includes the step of implementing consistency in an AP mode of the real-time operational DBMS by implementing the following steps. The method adds a set of schemes that enable a real-time operational DBMS to linearize read/write operations in all situations except a first specified situation and a second specified situation. The real-time operational DBMS is in AP mode, at least one master node for every data item is available in the database cluster of the real-time operational DBMS at all times. The method implements a CP mode of operation.

Methods and systems of managing consistency and availability tradeoffs in a real-time operational DBMS

A computer-implemented method of a distributed database system includes generating a database index. The method includes mapping a first specified number of bits of the database index to a database key. The method includes mapping a second specified number of bits to a data object associated with the database key. The method includes storing the first specified number of bits of the database key in a dram memory. The method includes storing second specified number of bits with the data object in a solid-state device (SSD) storage.

Methods and systems of splitting database indexes and digests

In one aspect, a computerized-method of a distributed database includes the step of providing a first data partition with a set of data. The method includes the step of providing a second data partition with a replica of the first set of data. The method includes the step of identifying a client application accessing the first data partition and the second data partition. The method includes the step of performing a reclamation operation on the second data partition. The method includes the step of informing the client application that a read operation is prohibited during the reclamation operation.

Methods and systems of garbage collection and defragmentation in a distributed database

In one embodiment, a computer-implemented method of a distributed database system included the step of providing a query processor to receive a query request and to chart out a query execution plan based on the query request. The method includes the step of providing a data generator to generate a digest list based on the query request, wherein the list of digests comprises each stream or filter applied in servicing the query request. The method includes the step of providing a query engine to obtain a digest list from a multiple-data source as a result of the query request and to filter the digest list. The method includes the step of providing a query worker to obtain the query request and process the query request based on a type of the query request. The method includes the step of providing one or more query worker threads to perform an asynchronous I/O operation for the query request.

Methods and systems of query engines and secondary indexes implemented in a distributed database

In one exemplary embodiment, a method includes allocating an arena block of a shared memory of a database node server. The arena block is divided into one or more slots. The one or more slots include a discreet and constant area of memory within the arena block. Each slot is assigned a constant-memory address relative to an arena-block's shared memory address. The index is implemented as a red-black tree data structure. Each red-black tree node is mapped to a slot. Each red-black-tree node is provided a pointer to one or more neighbor nodes. The index stored in shared memory can be used during a 'warm' rebooting process.

Method and system of implementing a database index in shared memory

For providing synchronization of image data among diverse devices and applications, exemplary systems include a direct client application providing a user interface operable to allow a user to organize image data according to albums, and to select one or more albums for synchronization with one or more server interfaces that provide various image data manipulation, sharing, and features related thereto. Systems also include an intermediary system to assist in synchronization of select albums with handheld devices, and a notification server to provide scalable notifications of album updates made at server interfaces by users. Exemplary methods include steps providing for selection, from direct client applications, of albums to synchronize with devices, including handheld devices, and various steps of providing updates and notifications among appropriate system components.

Synchronizing image data among applications and devices

In one exemplary aspect, a method of a distributed database system includes the step of detecting that a database node departed from a database cluster. A consensus-based voting process is implemented utilizing all of a set of other database nodes remaining in the database cluster. A cluster configuration is determined with the consensus-based voting process. The cluster configuration includes a new list of the set of other database nodes remaining in the database cluster. The data is automatically rebalanced among the set of other database nodes remaining in the database cluster according to the cluster configuration. Optionally, the consensus-based voting process can include a Paxos algorithm. The database cluster can be a Not-Only SQL (NOSQL) distributed database cluster.

Method and system of self-managing nodes of a distributed database cluster with a consensus algorithm

In one exemplary aspect, a method includes building a data index from a set of data from a client application in a distributed database system. The data index is optimized for operation in a DRAM data-storage device, and wherein the data index is built to use a set of fix-sized blocks that fit a DRAM device block size. The data index is maintained in the DRAM data-storage device. An SSD data-storage device is used to persistently store the data referenced by the data index in the DRAM data-storage device, and wherein the data and the metadata are stored in the SSD data-storage device in a log-based format.

Hybrid DRAM-SSD memory system for a distributed database node

In one exemplary aspect, a method of a cluster-node load balancing system of a distributed database system includes receiving a request from a cluster with at least one node of a cluster of the distributed database system. The request includes a query for an identity of all other nodes known by the node as well as a metadata of all data maintained by the node. The identity of all other nodes known by the node as well as the metadata of all data maintained by the node is provided to the cluster.

Cluster-node load balancing in a distributed database system

In one exemplary embodiment, a method of a distributed database system includes the step receiving a query in a query language from a client with a distributed database system. An index that matches the query is located. The index is pre-generated from a database table in the distributed database system. A map function of a MapReduce programming model is implemented using the index. A reduce function of the MapReduce programming model is implemented using the output of the map function. Optionally, a finalize function can be implemented using the output of the reduce function. The distributed database system can be a scalable NoSQL database. The reduce function can be optional when the value of the output of the map function is guaranteed to be unique.

Method and system of mapreduce implementations on indexed datasets in a distributed database environment

In one exemplary embodiment, a method of a distributed database system includes the step of receiving a database transaction with a node of the distributed database system. A priority of the database transaction is determined. A load of a transaction queue of the node is determined. The execution of a database transaction is delayed if the load of the transaction queue is greater than a first-water mark. The database transaction is delayed for a specified period. A portion of the database transaction is execution after the specified period. It is determined if the load of the queue is below a second-water mark after the specified period. A remaining portion of the database transaction can be processed if the load of the queue is below the second-water mark.

Growing community of inventors

Sunnyvale, CA, United States of America

Venkatachary Srinivasan