Sybase, Inc.

Google Inc.

Microsoft Technology Licensing, LLC

Xiaobin Ma

Xun Cheng

Peter J Schneider

Anjan Kumar Amirishetty

Viral Shah

Srinivasan Mottupalli

Prabhas Kumar Samanta

Scott M Meyer

Andrew Rodriguez

Walaa Eldin M Moustafa

Haoyu Huang

Aspects of the disclosure are directed to early filtering of candidate rows for a join operator of a query statement before the join operator is evaluated to generate a result set. Early filtering, e.g., before execution of the join operator, reduces the number of candidate rows fetched from a database during a join operator, which can improve the rate at which queries including join operators are executed by a DBMS for the database. One or more Bloom filters or other probabilistic data structures can be produced and consumed according to different join conditions, allowing for early pruning of unqualified rows during a database scan operation.

Database join operations with early filtering

A hybrid query execution plan is generated for executing queries on database systems implementing a columnar cache. A hybrid query execution plan combines a query execution plan for querying and retrieving data from a columnar cache and a base table. A columnar cache stores cached data in column-major format, which is logically represented by the database management system in row-major format. A database management system as described herein can scan valid blocks of column data according to a column scan operation. The system can identify invalidated blocks and execute a different sub-execution plan of the hybrid query execution plan to scan corresponding rows in tables corresponding to the location of data for the invalidated blocks.

Columnar cache query using hybrid query execution plan

Aspects of the disclosure are directed to late materialization of attributes in response to queries to a database implementing a database cache. Queried data is materialized in temporary memory before the data is projected as part of generating a result to the query. Instead of materializing all of the attributes referenced in a query before executing the query, a database management system materializes attributes as 'late' as possible—when the operation needing the attributes is executed. The operation needing the attributes can be performed sooner, as opposed to materializing all referenced attributes are materialized before executing the query.

Late materialization of queried data in database cache

Aspects of the disclosure provide for natively executing row-store expression data structures on column-store databases without rewriting. A database management system (DBMS) configured as described herein can maintain a mapping of row-store results to addresses of where corresponding column data is stored. When executing operators, such as logical operators, comparison operators, and/or function operators of a received query expression, the DBMS can operate on the column data, rather than the individual rows. The DBMS can store the results generated by executing the column operators, for example on a stack, and record the row-store addresses to which the stored results correspond. The DBMS responds with a number of rows corresponding to the processed column data.

Evaluating row-store expressions on a column-store database

Aspects of the disclosure are directed to generating a hybrid query execution plan for executing queries on database systems implementing a columnar cache. A hybrid query execution plan combines a query execution plan for querying and retrieving data from a columnar cache and a base table. A columnar cache stores cached data in column-major format, which is logically represented by the database management system in row-major format. A database management system as described herein can scan valid blocks of column data according to a column scan operation. The system can identify invalidated blocks and execute a different sub-execution plan of the hybrid query execution plan to scan corresponding rows in tables corresponding to the location of data for the invalidated blocks.

An original query execution plan of a database query is received. The original query execution plan represents a tree of operators. Source code for the original query execution plan is generated by a single traversal of the tree of operators. The generated source code is compiled into native machine code. The native machine code represents a simplified native access plan (SNAP).

Generating a producer-driven execution plan from a consumer-driven iterator-based execution plan

Disclosed herein are system, method, and computer program product embodiments for generating a native access plan for semi join operators. An embodiment operates by generating a plurality of variables based upon the positions of a plurality of operators in a compiled query plan, opening and traversing tables as the query plan is executed, and closing those tables based on the rows queried and the plurality of variables.

Generating a native access plan for semi join operators

The disclosed embodiments provide a system for processing queries of a graph database. During operation, the system executes a set of processes for maintaining the graph database storing a graph, wherein the graph includes a set of nodes, a set of edges between pairs of nodes in the set of nodes, and a set of predicates. When a write query of the graph database is received, the system uses one or more of the processes to process the write query by using a schema of the graph database to validate the write query. Next, the system transmits the validated write query to a set of shards containing the graph database. Finally, the system processes the write query at each shard when a shard identifier representing the shard is matched to one or more attributes in the write query.

Distributed graph database writes

Disclosed herein are system, method, and computer program product embodiments for generating a native access plan from a query execution plan for dynamic entity calling. An embodiment operates by receiving the query execution plan comprising at least one call to an entity, the entity being implemented by a plurality of classes, and generating source code of a native access plan that implements the query execution plan. The source code of the native access plan includes instructions to translate a run-time call to the entity to a call to a corresponding implementation of the entity based on an identifier of the called implementation of the entity.

Generating a native access plan for dynamic entity calling

Disclosed herein are system, method, and computer program product embodiments for eliminating redundancy when generating intermediate representation code. An embodiment operates by traversing a query execution plan, and for at least one operator in the query execution plan, determining whether the operator is derived from a parent class operator. If it is determined that the operator is derived from the parent class operator, source code for the native access plan is generated using one or more code generator functions corresponding to the parent class operator and/or one or more generator functions specifically corresponding to the child class operator. If it is determined that the operator is not derived from the parent class operator, source code for the native access plan is generated using one or more code generator functions corresponding to the operator.

Eliminating redundancy when generating intermediate representation code

A method can include receiving a request to execute a database command identifying a target table; identifying a plurality of rows to insert into the target table based in part on the database command; writing rows, from the plurality of rows, into a data page until the data page is full; determining, by an index thread manager, a number of threads to use for updating indexes defined for the target table; and upon determining the data page is full, updating, in parallel, the indexes defined for the target table using the number of threads.

Index updates using parallel and hybrid execution

Various embodiments of systems and methods to generate native access plan source code are described herein. In one aspect, a database query is received. A query execution plan, including a parent operator and one or more descendent operators, corresponding to the database query is retrieved. Further, a check is made to determine whether the parent operator and the one or more descendent operators include at least one loop. When both the parent operator and the one or more descendent operators include at least one loop, consume points for the at least one loop are defined. The parent operator and the one or more descendent operators are merged based on consume point types to generate native access plan source code.

Native access plan source code generation

A method can include initiating execution of a database command, the database command associated with a base table with at least one row to copy to a target table, the database command associated with a non-bulk insert mode; making a run-time decision on whether to automatically convert the insert mode from the non-bulk insert mode to a BULK insert mode based on the number of row buffers filled with rows from the base table during execution of the database command; and inserting at least one row into the target table using an insert mode based on the run-time decision.

Run-time decision of bulk insert for massive data loading

A method for inserting rows into a target table can include receiving a database command, the database command associated with a base table with at least one row to copy to a target table; receiving an indication that use of a BULK insert mode is feasible for the database command; based on the indication, and determining that an insert mode for the database command has been converted from a non-bulk insert mode to the BULK insert mode: reading a row from the base table; building the row read from the base table into an allocated row buffer; inserting the row into the target table in the BULK insert mode; and if it is determined that the allocated row buffer is full, updating at least one index in parallel with the inserting.

Growing community of inventors

Fremont, CA, United States of America

Xiaobin Ma