Google Inc.

Oracle International Corporation

Hewlett-packard Development Company, L.p.

Viral Shah

Anjan Kumar Amirishetty

Xun Cheng

Xiaobin Ma

Shasank Kisan Chavan

Amit Ganesh

Juan R Loaiza

Tirthankar Lahiri

Kothanda Umamageswaran

Jesse Kamp

Krishnan Meiyyappan

Hariharan Lakshmanan

Xia Hua

Anoop Sharma

Adrian Tsz Him Ng

Roger Dermot Macnicol

Maria Colgan

Kamaljit Shergill

Sudhir Radhakrishna Shetiya

Lars B Plum

Yiran Qin

Haoyu Huang

Roger D Macnicol

A hybrid query execution plan is generated for executing queries on database systems implementing a columnar cache. A hybrid query execution plan combines a query execution plan for querying and retrieving data from a columnar cache and a base table. A columnar cache stores cached data in column-major format, which is logically represented by the database management system in row-major format. A database management system as described herein can scan valid blocks of column data according to a column scan operation. The system can identify invalidated blocks and execute a different sub-execution plan of the hybrid query execution plan to scan corresponding rows in tables corresponding to the location of data for the invalidated blocks.

Columnar cache query using hybrid query execution plan

Aspects of the disclosure are directed to maintaining transaction consistency when using a columnar cache. The columnar cache can be initially loaded with all-visible data, and as the data gets modified, respective data is invalidated in the columnar cache. As more data gets invalidated in the columnar cache, respective data can be refreshed in the columnar cache. As part of the refresh, the latest all-visible data can be populated while the queries are still using the old data in the columnar cache. When all the queries transition to use the newly populated data, old data can be removed from the columnar cache. A query can use valid blocks of columnar cache and go to a row store for invalid blocks. When a query starts to use the columnar cache, a request can be submitted to asynchronously prefetch the invalid blocks from the row store.

Maintaining transactional consistency in columnar engine

Aspects of the disclosure are directed to late materialization of attributes in response to queries to a database implementing a database cache. Queried data is materialized in temporary memory before the data is projected as part of generating a result to the query. Instead of materializing all of the attributes referenced in a query before executing the query, a database management system materializes attributes as 'late' as possible—when the operation needing the attributes is executed. The operation needing the attributes can be performed sooner, as opposed to materializing all referenced attributes are materialized before executing the query.

Late materialization of queried data in database cache

A method for providing elastic columnar cache includes receiving cache configuration information indicating a maximum size and an incremental size for a cache associated with a user. The cache is configured to store a portion of a table in a row-major format. The method includes caching, in a column-major format, a subset of the plurality of columns of the table in the cache and receiving a plurality of data requests requesting access to the table and associated with a corresponding access pattern requiring access to one or more of the columns. While executing one or more workloads, the method includes, for each column of the table, determining an access frequency indicating a number of times the corresponding column is accessed over a predetermined time period and dynamically adjusting the subset of columns based on the access patterns, the maximum size, and the incremental size.

Elastic columnar cache for cloud databases

Aspects of the disclosure provide for natively executing row-store expression data structures on column-store databases without rewriting. A database management system (DBMS) configured as described herein can maintain a mapping of row-store results to addresses of where corresponding column data is stored. When executing operators, such as logical operators, comparison operators, and/or function operators of a received query expression, the DBMS can operate on the column data, rather than the individual rows. The DBMS can store the results generated by executing the column operators, for example on a stack, and record the row-store addresses to which the stored results correspond. The DBMS responds with a number of rows corresponding to the processed column data.

Evaluating row-store expressions on a column-store database

Aspects of the disclosure are directed to generating a hybrid query execution plan for executing queries on database systems implementing a columnar cache. A hybrid query execution plan combines a query execution plan for querying and retrieving data from a columnar cache and a base table. A columnar cache stores cached data in column-major format, which is logically represented by the database management system in row-major format. A database management system as described herein can scan valid blocks of column data according to a column scan operation. The system can identify invalidated blocks and execute a different sub-execution plan of the hybrid query execution plan to scan corresponding rows in tables corresponding to the location of data for the invalidated blocks.

Techniques are provided for a DBMS automating ILM on indexes, based on index composition, to efficiently reduce index storage footprints. According to an embodiment, a user sets an index-specific ILM (ISILM) policy, which comprises one or both of an index-test requirement and a time requirement. Based on the ISILM policy being met, or on some other way of initiating analysis, the DBMS automatically analyzes the data blocks storing the index to determine an index condition metric (e.g., percentage of free space). This analysis is performed on a sample of data blocks storing the index without blocking the index from other operations during the analysis. The condition metric for the entire index is estimated based on analysis of the sample data blocks. Using the determined condition metric for an index, the DBMS automatically selects an option for optimally managing the index (e.g., coalesce, shrink space, index rebuild, no action, etc.).

Automated information lifecycle management of indexes

Techniques are provided for storing in in-memory unit (IMU) in a lower-storage tier and copying the IMU to DRAM when needed for query processing. Techniques are also provided for copying IMUs to lower tiers of storage when evicted from the cache of higher tiers of storage. Techniques are provided for implementing functionality of IMUs within a storage system, to enable database servers to push tasks, such as filtering, to the storage system where the storage system may access IMUs within its own memory to perform the tasks. Metadata associated with a set of data may be used to indicate whether an IMU for the data should be created by the database server machine or within the storage system.

Columnar caching in tiered storage

There is provided a computer-executed method of freeing memory. One exemplary method comprises receiving a message from a user process. The message may specify a virtual address for a memory segment. The virtual address may be mapped to the memory segment. The memory segment may comprise a physical page. The method may further comprise identifying the physical page based on the virtual address. Additionally, the method may comprise freeing the physical page without unmapping the memory segment.

Growing community of inventors

Cupertino, CA, United States of America

Viral Shah