Microsoft Technology Licensing, LLC

Adrian Ilcu Predescu

Dharma Kiritkumar Shukla

Karthik Raman

Momin Mahmoud Al-Ghosien

Shireesh Kumar Thota

Pankaj Sharma

Madhan Gajendran

Ankur Savailal Shah

Mikhail Mikhailovich Koltachev

Hari Sudan Sundar

Samer Boshra

Liang Li

Krishnan Sundaram

Brandon Chong

Aravind Ramachandran Krishna

Karan Vishwanath Popali

Amir Netz

Cristian Petculescu

Marius Dumitru

Jayanta Mondal

Siddhesh Dilip Vethe

Oliver Drew Leonard Towers

Vinod Sridharan

Orestis Kostakis

Balachandar Perumalswamy

Vivek Ravindran

Ashwini Singh

Emily Lawton

Data sets such as databases are often distributed over a number of servers, where each server stores a subset of records and an index that enables the server to locate the records in response to queries. However, tight coupling of indexing and storage may limit load-balancing, fault recovery, and distribution. Instead, the set of servers may be partitioned into a set of storage servers that store the records and a set of index servers of the index over the records. In a set that is decoupled in this manner, load-balancing may involve provisioning and locating index servers independently of the provisioning and locating of the storage servers in view of the particular sensitivities and tolerances of various applications. Additionally, the index servers may also utilize a data layout that is selected and adapted independent of the data layout of the storage servers and/or the schema of the data stored thereby.

Schema-agnostic indexing of distributed databases

Databases are often provided according to various organizational models (e.g., document-oriented storage, key/value stores, and relational database), and are accessed through various access models (e.g., SQL, XPath, and schemaless queries). As data is shared across sources and applications, the dependency of a data service upon a particular organizational and/or access models may become confining. Instead, data services may store data in a base representation format, such as an atom-record-sequence model. New data received in a native item format may be converted into the base representation format for storage, and converted into a requested format to fulfill data requests. Queries may be translated from a native query format into a base query format that is applicable to the base representation format of the data set, e.g., via translation into an query intermediate language (such as JavaScript) and compilation into opcodes that are executed by a virtual machine within the database engine.

Unified data organization for multi-model distributed databases

The subject disclosure relates to querying of column based data encoded structures enabling efficient query processing over large scale data storage, and more specifically with respect to complex queries implicating filter and/or sort operations for data over a defined window. In this regard, in various embodiments, a method is provided that avoids scenarios involving expensive sorting of a high percentage of, or all, rows, either by not sorting any rows at all, or by sorting only a very small number of rows consistent with or smaller than a number of rows associated with the size of the requested window over the data. In one embodiment, this is achieved by splitting an external query request into two different internal sub-requests, a first one that computes statistics about distribution of rows for any specified WHERE clauses and ORDER BY columns, and a second one that selects only the rows that match the window based on the statistics.

Efficient large-scale filtering and/or sorting for querying of column based data encoded structures

Growing community of inventors

Sammamish, WA, United States of America

Adrian Ilcu Predescu