Microsoft Technology Licensing, LLC

Krystyna Ewa Reisteter

Cristian Diaconu

Naveen Prakash

Alejandro Hernandez Saenz

Alexander Budovski

Rogério Ramos

Huanhui Hu

Hanumantha Rao Kodavalla

Sheetal Shrotri

Peter Byrne

Sandeep Lingam

Rogerio Ramos

Siddharth Deepak Mehta

Sarika R Iyer

Vikram Wakade

Distributed database systems including compute nodes and page servers are described herein that enable separating logical and physical storage of database files in a distributed database system. A distributed database system includes a page server and a compute node, and is configured to store a logical database file that includes data and is associated with a file identifier. Each page server is configurable to store slices (i.e., subportions) of the logical database file. The compute node is coupled to the plurality of page servers and configured to store the logical database file responsive to a received command. In an aspect, such storage may comprise slicing the data comprising the logical database file into a set of slices with each being associated with a respective page server, maintaining an endpoint mapping for each slice of the first set of slices, and transmitting each slice to the associated for storage thereby.

Separation of logical and physical storage in a distributed database system

Data base performance is improved using write-behind optimization of covering cache. Non-volatile memory data cache includes a full copy of stored data file(s). Data cache and storage writes, checkpoints, and recovery may be decoupled (e.g., with separate writes, checkpoints and recoveries). A covering data cache supports improved performance by supporting database operation during storage delays or outages and/or by supporting reduced I/O operations using aggregate writes of contiguous data pages (e.g., clean and dirty pages) to stored data file(s). Aggregate writes reduce data file fragmentation and reduce the cost of snapshots. Performing write-behind operations in a background process with optimistic concurrency control may support improved database performance, for example, by not interfering with write operations to data cache. Data cache may store (e.g., in metadata) data cache checkpoint information and storage checkpoint information. A stored data file may store storage checkpoint information (e.g., in a file header).

Write-behind optimization of covering cache

Distributed database systems including compute nodes and page servers are described herein that enable separating logical and physical storage of database files in a distributed database system. A distributed database system includes a page server and a compute node and is configured to store a logical database file that includes data and is associated with a file identifier. Each page server is configurable to store slices (i.e., subportions) of the logical database file. The compute node is coupled to the plurality of page servers and configured to store the logical database file responsive to a received command. In an aspect, such storage may comprise slicing the data comprising the logical database file into a set of slices with each being associated with a respective page server, maintaining an endpoint mapping for each slice of the first set of slices, and transmitting each slice to the associated for storage thereby.

Embodiments described herein detect data corruption in a distributed data set system. For example, a system comprises node(s) for processing queries with respect to a distributed data set comprising a plurality of storage segments. A write transaction resulting from a query with respect to a particular storage segment is logged in a log record that describes a modification to the storage segment. A log service provides the log record to a data server managing a portion of the distributed data set in which the storage segment is included, which performs the write transaction with respect to the storage segment. For redundancy purposes, the data server has replica(s) that manage respective replicas of the portion of the distributed data set managed thereby. For backup purposes, snapshots of the replica(s) are periodically generated. To determine a data corruption, a snapshot of one replica is cross-validated with a snapshot of another replica.

Snapshot-based data corruption detection

A storage segment server environment that includes storage segment server(s) that cover a contiguous address space of data. The storage segment server has storage segments that are contiguous and that matches the address space of a respective of storage segments in the cloud store service. Thus, a read of the contiguous address spaces of the data from the cloud store service may instead be handled as a single read of the contiguous address space of the storage segment server. Each storage segment server(s) may be assigned to a range within the total contiguous address space of the data. Each storage segment server(s) includes a respective storage segment write component that is configured to perform write operations to storage segments in the respective storage segment server. Each storage segment server(s) also includes a respective cloud storage write component that initiates corresponding write operations to the cloud store service.

Storage segment server covered cache

Use of a log service computing system to apply a log to storage segments in the context of there being a primary compute system that operates upon data, and secondary compute system(s) that are at the ready to be promoted to the new primary compute system should the old primary compute system fail. Rather than have each of the primary and secondary compute systems maintain their own replicated of the entire data locally, the data is separated in storage segments that are collectively distributed amongst multiple storage segment servers.

Growing community of inventors

Kirkland, WA, United States of America

Krystyna Ewa Reisteter