Microsoft Technology Licensing, LLC

Zichen Sun

Shane Kumar Mainali

Maneesh Sah

Quan Zhang

Shaoyu Zhang

Georgi M Chalakov

Krishnan Varadarajan

Jun Tse Chen

Thomas Leo Marquardt

Saurabh Pant

Dana Yulian Kaban

Saher B Ahwal

Da Zhou

Amit Pratap Singh

Junhua Gu

Esfandiar Manii

Venkates Paramasivam Balakrishnan

Jegan Devaraju

Jingchao Zhang

Wei Zhu Chen

Ju Wang

Wei Ting Chen

Manish Chablani

Hao Feng

Arild Einar Skjoldsvold

Jianwu Gao

Kaifeng Zhu

Andrew James Edwards

Michael E Roberson

Jieqing Wang

Wei Chen

Advait Kumar Mishra

Jun Chen

A distributed storage system includes table controllers collectively storing partitions of a logical table having multiple rows. For each of the partitions, a corresponding table controller stores a subset of the rows of the logical table across individual pages. A management controller updates a schema of the logical table from a first to a second schema definition by, for each of the partitions, sending an update command indicating the second schema definition to the corresponding table controller. The first table controller corresponds to a first partition that includes a first subset of the rows. The first table controller, prior to receiving the update command, includes an identifier of the first schema definition within each page of data for the first subset. Subsequent to receiving the update command, the first table controller, when writing each new page of data for the first subset, includes an identifier of the second schema definition.

Online schema change of range-partitioned index in a distributed storage system

A service enables a command that refers to a file system object using a hierarchical namespace identifier to be executed against the file system object in a flat namespace. The service selectively distributes the command to one of a plurality of name resolution nodes based on a directory name included in the hierarchical namespace identifier. The identified node resolves the directory name to a flat namespace identifier that is used to execute the command against the flat namespace. After communicating with at least one storage node to resolve a directory name, each name resolution node stores a mapping of the directory name to the corresponding flat namespace identifier in a cache, so that subsequent resolutions of that directory name may be performed more efficiently. Cache entries may be invalidated when an operation occurs that impacts the relevant mapping and/or based on system considerations such as cache expiry.

Hierarchical namespace service with distributed name resolution caching and synchronization

A snapshot of data from a table associated with a particular user may be generated. Tree data structures that are distributed across multiple computer systems may be accessed. Each of the tree structures may include data associated with one or more users. At least one tree data structure of the tree data structures that includes data associated with the particular user of the one or more users may be identified. The at least one tree data structure may then be filtered. Filtering may comprise identifying only data that is associated with the particular user. A snapshot of the data associated with the particular user may be generated. Generating the snapshot of the data associated with the particular user comprises generating a data structure that is configured to map to each data page of the at least one tree data structure that includes data associated with the particular user.

Efficient snapshot generation of data tables

In various embodiments, methods and systems for implementing hash-based partitioning in distributed computing systems are provided. At a high level, a distributed computing system having an underlying range-based partitioning architecture for storage may be configured as a hash-based partitioning system, for example, a hybrid range-hash table storage. An operations engine of the hash-based partitioning system receives a tenant request to provision input/output operations per second (IOPS). The tenant request comprises a requested number of IOPS. Based on the tenant request, a provisioning operation to provision IOPS in a hybrid range-hash table storage with hash-based partitioning is determined. The provisioning operation is selected from one of the following: a table creation provisioning operation, an IOPS increase provisioning operation, and an IOPS decrease provisioning operation. The selected provisioning operation is executed for a corresponding table. A user request for data is processed using the table associated with the requested number of IOPS.

Hash-based partitioning system

Methods, systems, and computer programs are presented for providing file system functions on a cloud storage system based on blob storage. A cloud storage system comprises storage clusters, and each storage cluster comprises a plurality of storage nodes (each including a partition layer that stores data in blobs organized within containers, and a stream layer for streaming blob data), a hierarchical namespace service for implementing file system functions on the data stored in the blobs and, a front end layer for processing user requests to access and manage data stored on the blobs. The file system functions include flat namespace functions and hierarchical namespace functions. Further, the front end layer accesses the partition layer to access the blobs based on addresses of blobs and accesses the hierarchical namespace service to identify the blobs.

Cloud storage distributed file system

Methods, systems, and programs provide for executing distributed transactions in a cloud storage system with a hierarchical namespace. One method includes receiving a request with operations to be executed atomically. Further, nodes are identified for executing the operations, each node having a respective clock and having at least part of a transactions table for controlling updates to entities. Each clock is one of a loosely-synchronized, a strictly-synchronized clock, a logical, or a physical clock. Additionally, the nodes process the operations, which includes setting a commit timestamp (CS) to a value of the clock in the node if the node is a first node in the processing. One node coordinates the transactions, and may be one of the nodes executing transactions. If the clock in the node is less than a current value of the CS, the node waits for the clock to reach the current value of the CS and the CS is updated. The transactions table is updated based on the value of the CS, the atomic execution is committed utilizing the final value of the CS, and a status is returned.

Distributed transactions in cloud storage with hierarchical namespace

Methods, systems, and apparatuses are provided for a storage system that implements a hierarchical namespace service. A storage system includes a plurality of physical nodes and a plurality of sets of virtual nodes. Each set of virtual nodes is managed by a corresponding physical node. Each virtual node is configured to manage a respective set of directory blocks. Each directory block is a respective partition of a storage namespace and is managed by a corresponding single virtual node. Each virtual node maintains a directory block map. The directory block map maps file system object names in a hierarchical namespace to entity block identifiers in the flat namespace for entity blocks (files and folders) stored in directories corresponding to the managed set of directory blocks. Load balancing may be performed by moving virtual nodes between physical nodes, and by splitting directory blocks.

Hierarchical namespace with strong consistency and horizontal scalability

Methods for dequeue optimizations in queues are performed by systems and apparatuses. The methods optimize dequeue operations using aggregation of expired messages enqueued in a queue and conditional iteration over enqueued messages based on the aggregation to service dequeue commands. Queues utilize page hierarchies such as root pages, index pages, and data pages. The aggregation of expired messages for pages in the queue determines the latest expired time for messages for a given page, and these latest expired times are stored in their respective pages, including data pages, index pages, and root pages, for use in the conditional iteration. The conditional iteration bypasses pages for which a latest expired time for all messages is prior to the current time when servicing dequeue requests for the queue.

System and method for dequeue optimization using conditional iteration

Growing community of inventors

Sammamish, WA, United States of America

Zichen Sun