Vmware, Inc.

Kapil Chowksey

Abhishek Gupta

Richard Paul Spillane

Jorge Guerra Delgado

Srinath Premachandran

Sandeep Rangaswamy

Wenguang Wang

Robert T Johnson

Rob T Johnson

Ittai Abraham

Exemplary methods, apparatuses, and systems include a file system process obtaining locks on a first node and a second node in a tree structure, with the second node being a child node of the first node. The file system process determines a quantity of child nodes of the second. While holding the locks on the first and second nodes, the file system determines whether to proactively split or merge the second node. In response to determining that the quantity of child nodes is within a first range, the file system process splits the second node. If the file system process determines that the quantity of child nodes is within a second range, the file system process merges the second node.

Proactive splitting and merging of nodes in a B-tree

Exemplary methods, apparatuses, and systems include a file system process reading a first node in a tree data structure from a first memory. The first node includes a first approximate membership query data structure ('AMQ'), a first plurality of child pointers, a first plurality of pivot values, and a first buffer. The file system process determines that the first plurality of child pointers exceeds a maximum size. Using a pivot value in the first plurality of pivot values, the file system process splits the first node into a second node and a third node. The file system process uses the pivot value to split the first buffer into a second buffer and a third buffer. Using the pivot value and the first AMQ, the file system process generates a second AMQ and a third AMQ.

Reduced bandwidth queries in B e-trees using approximate membership query data structures

Exemplary methods, apparatuses, and systems include a file system process inserting a first key/value pair and a second key/value pair into a first tree. The second key is a duplicate of the first key and the value of the second key/value pair is an operation changing the value. In response to a request for a range of key/value pairs, the process reads the second key/value pair and inserts it in a second tree. The process reads the first pair and determines, while inserting the first pair in the second tree, that the second key is a duplicate of the first key. The file system process determines an updated value of the first value by applying the operation in the second value to first value. The file system operation updates the second key/value pair in the second tree with the updated value and returns the requested range of key/value pairs.

Range lookup operations for B-trees using update messages

Embodiments described herein are related to a method of scanning a B-tree. For example, a method comprises receiving a scan request to scan a B-tree having a plurality of levels, each level comprising one or more nodes, wherein for each of one or more levels of the plurality of levels, nodes are grouped into groups, where nodes of any given group are stored across sequential disk blocks. The method further comprises generating a queue for each level of the B-tree. For each queue, the method further comprises loading into memory a next group of nodes based upon determining a storage location of a node of the next group of nodes.

Read iterator for pre-fetching nodes of a B-tree into memory

Embodiments of the present disclosure relate to techniques for performing a merge update for a database. In particular, certain embodiments of a method include generating a message comprising a first key and a first transaction associated with the first key, the first transaction indicating a transaction to perform other than for key-value pairs comprising the first key. The method further includes storing the message in a database. The method further includes merging the message with a first key-value pair stored in the database, the first-key value pair comprising the first key. The method further includes performing the first transaction based on merging the message with the first key-value pair.

Merge updates for key value stores

Exemplary methods, apparatuses, and systems include a file system process determining to a flush a node in a first tree. The first node includes a buffer structured as a second tree. The file system process generates an input/output instruction to load the buffer from a first memory to a second memory. The second tree is stored in two more non-contiguous locations in the first memory and the input/output operation includes a read operation corresponding to each of the two or more non-contiguous locations. The file system process causes the input/output instruction to be executed concurrently on the first memory.

Optimizing operations on b e-trees stored on solid state media

A B-tree associated with a file system on a storage volume includes a hierarchy of nodes. Each node includes a buffer portion to store key-value pairs as messages in the buffer. Each node can be characterized by having a maximum allowable size that is periodically updated at run time. The buffers in the nodes of the B-tree are therefore characterized by having a maximum allowed size that can vary over time.

Auto-tuned write-optimized key-value store

Embodiments herein are directed towards systems and methods for performing range lookups in B-trees. One example method involves receiving a request to return key-value pairs within a range of keys from the B-tree. The B-tree includes a plurality of nodes, each node being associated with a buffer that stores key-value pairs. The method further involves determining a fractional size of the range of keys. The method further involves, for each level of the B-tree, obtaining from within one or more buffers of one or more nodes of the level, a set of key-value pairs within the range of keys up to a size equal to the fractional size and transferring the set of key-value pairs to a result data structure. The method further involves sorting and merging all key-value pairs in the result data structure and returning the result data structure in response to the request.

Large range lookups for B-tree

A buffer tree structure includes, at each internal node, a buffer having a compacted portion and an uncompacted portion. Insertion of data elements to the buffer tree can occur units called packets. A packet is initially stored in the uncompacted portion of a receiving node's buffer. When a compaction trigger condition exists, packet compaction is performed including a data element compaction operation. A buffer-emptying (flush) operation pushes the compacted packets to children nodes.

Bounding cost of flushes in buffer trees

Embodiments described herein are related to bulk loading data into a B-tree. Embodiments include generating a first leaf node of a B-tree by allocating a first page for the first leaf node from a leaf page queue comprising a first plurality of sequential pages; and writing one or more tuples to the first page allocated for the first leaf node. Embodiments further include generating an parent node for the first leaf node and a second leaf node of the B-tree by allocating a third page for the parent node from an parent page queue comprising a second plurality of sequential pages, the parent node comprising a first indication of the first leaf node and a second indication of the second leaf node, the first indication and the second indication stored in the third page allocated for the parent.

Bulk-load for B-trees

Embodiments described herein involve improved management of snapshots of a file system. Embodiments include copying a first root node of a first snapshot to a second snapshot, the second snapshot referencing other nodes of the first snapshot. Embodiments further include incrementing reference counts of the other nodes of the first snapshot. Embodiments further include adding a storage address of the first root node to a list. Embodiments further include, each time that a copy on write operation is performed for a node of the other nodes, adding a storage address of the node to the list and decrementing the reference count of the node. Embodiments further include iterating through the list and, for each storage address in the list, decrementing the reference count of the node corresponding to the storage address and, if the reference count of the node reaches zero, freeing storage space at the storage address.

Optimal snapshot deletion

The subject matter described herein is generally directed towards tracking and recovering a disk allocation state. An on-disk log of operations is maintained to describe operations performed to an in-memory partial reference count map. Upon a crash of a host computing device during a checkpoint operation to an on-disk complete reference count map, the on-disk log of operations is used to undo and then redo the operations, or just redo the operations. In this manner, a disk allocation state prior to the crash is recreated in the on-disk complete reference count map with atomicity and crash consistency.

Tracking and recovering a disk allocation state

A buffer tree structure includes, at each internal node, a buffer having a compacted portion and an uncompacted portion. Insertion of data elements to the buffer tree can occur units called packets. A packet is initially stored in the uncompacted portion of a receiving node's buffer. After a time, packets in the uncompacted portion of a buffer are combined into compacted packets in the compacted portion of the buffer. A buffer-emptying (flush) operation pushes the compacted packets to children nodes.

Reducing write amplification in buffer trees

Certain aspects provide systems and methods for performing an operation on a B-tree. A method comprises writing a message associated with the operation to a first slot in a first buffer of a first non-leaf node of the B-tree in an append-only manner, wherein a first filter associated with the first slot is used for query operations associated with the first slot. The method further comprises determining that the first buffer is full and, upon determining to flush the message to a non-leaf child node, flushing the message in an append-only manner to a second slot in a second buffer of the non-leaf child node, wherein a second filter associated with the second slot is used for query operations associated with the second slot. The method further comprises, upon determining to flush the message to a leaf node, flushing the message to the leaf node in a sorted manner.

Trading off cache space and write amplification for B-trees

Data storage system and method for managing transaction requests to the data storage system utilizes an active write ahead log and a standby write ahead log to apply the transaction requests to a storage data structure stored in a storage system of the data storage system.

Growing community of inventors

Cupertino, CA, United States of America

Kapil Chowksey