Microsoft Technology Licensing, LLC

Badrish Chandramouli

Jonathan David Goldstein

David Maier

Mike Barnett

Mohamed Hassan Ali

John Carlton Platt

Donald Kossmann

Roman Schindlauer

Yinan Li

Steven Mark Drucker

Balan Sethu Raman

Danyel Aharon Fisher

Michael Barnett

Robert Anthony DeLine

Peter Zabback

Surajit J Chaudhuri

Vivek Ravindranath Narasayya

Suman Kumar Nath

Justin Jon Levandoski

Sudipto Das

Manoj A Syamala

Feng Li

Abdul Hussain Quamar

Tomer Verona

James Felger Terwilliger

James Adam Hunter

Gunaprasaad Jeganathan

Nikolaos Romanos Katsipoulakis

Hyunjung Park

Wenchao Zhou

Lev Novik

Torsten Walther Grabs

Di Wang

Beysim Sezgin

Olivier Nano

Tihomir Tsvetkov Tarnavski

Songyun Duan

Colin Joseph Meek

Ivo José Garcia Dos Santos

Ciprian Gerea

Laurent Bussard

Ahmed Eldawy

Xin Jin

John Wernsing

Ping Wang

Raul Castro Fernandez

Galex Sunyul Yen

Asvin Ananthanarayan

Jianan Lu

Milos Nikolic

Peter A Zabback

Systems and methods of processing queries for column-oriented database systems involve processing the queries to identify which of values of column data are to be accessed for the query. A select bitmap is then generated having k bits wherein each bit corresponds to one of the values of the column. The select bitmap is generated such that each bit representing a value of the column data that is to be accessed for the query has a first value and each bit representing a value of the column data that is not to be accessed for the query has a second value. Encoded values are then extracted from memory for each of the values represented in the select bitmap by a bit having the first value. The extracted encoded values are decoded to generate decoded query data. The decoded query data is then processed to generate result data.

Selection pushdown in column stores using bit manipulation instructions

A method includes obtaining a query containing at least one field from which data is being queried, obtaining a dataset having a schema-free data exchange format having multiple fields of data at different physical positions in the dataset, and parsing the dataset by obtaining a structural index that maps logical locations of fields to physical locations of the fields of the dataset, accessing the structural index with logical locations of the fields that index to the physical locations, and providing data from the fields based on the physical locations responsive to the query.

Parser for schema-free data exchange format

A system includes a processor and a memory storing multiple records in a key-value data structure and processor executable instructions for managing access to records in the key-value data structure. The instructions are executable to perform operations including managing a shared atomic epoch counter and thread epoch counters, determining a maximal safe epoch as a function of the shared atomic epoch counter and the thread epoch counters, maintaining a drain list of trigger actions, and triggering the trigger actions in the drain list as a function of an update of the shared atomic epoch counter and the maximal safe epoch.

Key-value store system

The described implementations relate to processing of electronic data. One implementation is manifest as a technique that can include obtaining a relational query that references one or more data items and associating progress intervals with the data items. The technique can also include converting the relational query into a corresponding streaming query, and providing the streaming query and the data items with the progress intervals to a stream engine that produces incremental results of the query. For example, the progress intervals can be based on row numbers of a relational database table. The progress intervals can be used to define event lifetimes of streaming events that are provided as inputs to the stream engine.

Progressive query computation using streaming architectures

An intermediate representation of a workflow of one or more modules may be generated to decouple language implementations of the one or more modules. In response to receiving a workflow of one or more modules, the workflow may be analyzed to determine an optimal implementation language for each of the one or more modules to thereby reduce effects of data marshalling. An intermediate representation of the workflow that is configured to decouple any implementation languages associated with the one or more modules may be generated. To allow for decoupling, the intermediate representation may be written in a declarative language. The generated intermediate representation may then be compiled to generate an executable program that corresponds to the workflow and is implemented in the determined optimal language for each of the one or more modules.

Dynamic pattern matching code generation

Costs associated with deploying a streaming query according to one or more resiliency strategies given a particular service level agreement (SLA) specification are modeled to enable selection and/or recommendation of a particular resiliency strategy. A baseline cost model represents costs associated with deploying the streaming query non-resiliently. For each of any number of resiliency strategies, a resiliency model represents additional costs associated with deploying the streaming query according to a particular resiliency strategy.

Modeling resiliency strategies for streaming queries

Accommodating a particular kind of out-of-order data; namely, when data arrives out-of-order because the data is generated in systems that may have different application timelines. As data arrives, it arrives with a particular partition key. For instance, the partition key might identify the source of the data. The data from each partition key is kept in-order with respect to all other data from that same partition key. That said, data from one key is permitted to have a different timeline as compared to data from any other key. Thus, the data may not be in-order when viewed globally across keys. Rather, data is tolerated to be out-of-order globally so long as the lag in the arrived data is within some tolerated lag. If the data arrives having a time that exceeds the maximum tolerated lag, then the system applies some policy to determine what to do with the delayed data.

Stream data processing on multiple application timelines

High performance query processing and data analytics can be performed across architecturally diverse scales, such as single core, multi-core and/or multi-nodes. The high performance query processing and data analytics can include a separation of query computation, keying data, and data movement and parallel computation, thereby enhancing the capabilities of the query processing and data analytics, while allowing the specification of complex forms of data parallel computation that may execute across real-time and offline. The decoupling of data movement and parallel computation, as described herein can improve query processing and data analytics speed, can provide for the optimization of searches in a plurality of computing environments, and can provide the ability to search through a larger space of execution plans.

High performance query processing and data analytics

Architecture introduces a new pattern operator referred to as called an augmented transition network (ATN), which is a streaming adaptation of non-reentrant, fixed-state ATNs for dynamic patterns. Additional user-defined information is associated with automaton states and is accessible to transitions during execution. ATNs are created that directly model complex pattern continuous queries with arbitrary cycles in a transition graph. The architecture can express the desire to ignore some events during pattern detection, and can also detect the absence of data as part of a pattern. The architecture facilitates efficient support for negation, ignorable events, and state cleanup based on predicate punctuations.

Dynamic pattern matching over ordered and disordered data streams

The formulation of a merged sorted list from multiple input sorted lists in multiple phases using an array pair. Initially, the first array is contiguously populated with the input sorted lists. In the first phase, the first and second input sorted lists are merged into a first intermediary merged list within the second array. Each subsequent phase merges a prior intermediary merged list resulting from the prior phase and, a next input sorted list in the first array to generate a next intermediary merged list, or a merged sorted list if there or no further input in the first array. The intermediary merged lists alternate between the first array and the second array from one phase to the next phase.

Merging of sorted lists using array pair

Techniques for tenant performance isolation in a multiple-tenant database management system are described. These techniques may include providing a reservation of server resources. The server resources reservation may include a reservation of a central processing unit (CPU), a reservation of Input/Ouput throughput, and/or a reservation of buffer pool memory or working memory. The techniques may also include a metering mechanism that determines whether the resource reservation is satisfied. The metering mechanism may be independent of an actual resource allocation mechanism associated with the server resource reservation.

Isolating resources and performance in a database management system

The techniques and systems described herein are directed to incremental sorting in a stream processing engine. For example, because of network delays, streaming data can be received at a stream processing engine that is out-of-order with respect to the timestamp of the event generation. Streaming data can be incrementally sorted by partitioning events into ordered runs, which provide intermediate arrays of ordered elements. Upon an assertion of a punctuation event (e.g., a progress indicator), partial runs can be output and merged to form sorted streaming data. Further, the sorted streaming data can be used to provide speculative outputs based on speculation latencies, which improves throughput and completeness of processing on the streaming data.

Efficient sorting for a stream processing engine

Growing community of inventors

Redmond, WA, United States of America

Badrish Chandramouli