Microsoft Technology Licensing, LLC

Snowflake Inc.

Rishabh Singh Ahluwalia

Benoit Dageville

Subramanian Muralidhar

Nithin Mahesh

Srikumar Rangarajan

Sandeep Lingam

Yi Fang

Steven John Lindell

Lin Chan

Yiming Kang

Vikram Wakade

Tianhui Shi

Described herein are techniques for compact representation of table columns via templatization. Column templates can model the collection of columns in a table as a logical entity. The logical entity can be split into two objects. A first object can contain a subset of column attributes (e.g., fields) that can be shared with other tables, such as name and data type. A second object can contain another subset of column attributes that are unique to a table, such as time stamps and created-on information.

Compact representation of table columns via templatization

Methods for rowgroup consolidation with delta accumulation and versioning in distributed systems are performed. The systems provide performant methods of row storage that enable versioned modifications of data while keeping and allowing access to older versions of the data for point-in-time transactions. The accumulation of valid rows, deletes, and modifications is maintained in blobs for rowgroups until a size threshold is reached, at which point the rows are moved into a columnar compressed form. Changes to data and associated metadata are stored locally and globally via appends, maintaining logical consistency. Metadata is stored in footers of files allowing faster access to the metadata and its associated data for transactions and instant rollback via metadata version flipping for aborted transactions, as well as lock-free reads of data.

Rowgroup consolidation with global delta accumulation and versioning in distributed systems

Growing community of inventors

Bellevue, WA, United States of America

Rishabh Singh Ahluwalia