Snowflake Inc.

Microsoft Technology Licensing, LLC

Sahaj Saini

Subramanian Muralidhar

Nithin Mahesh

Eric Robinson

Shreyas Narendra Desai

Prasanna V Krishnan

German Alberto Gil Echeverri

Pui Kei Johnston Chu

Robert Bengt Benedikt Gernhardt

Benoit Dageville

Vishnu Dutt Paladugu

Di Wu

Chao-Yang Lo

Damien Carru

Saurin Shah

David Schultz

Polita Paulus

Mikhail Kazhamiaka

Chieh-Sheng Wang

Srinidhi Karthik Bisthavalli Srinivasa

Laxman Mamidi

Matthew Uhlar

Travis Austin Wright

Lin Chan

Stanislav A Oks

Hitesh Madan

Sarvesh Singh

Jasraj Dange

Krishna B Nibhanupudi

Weiyun Huang

Loc Duc Bui

Jarupat Jisarojito

Stuart Padley

Umachandar Jayachandran

William Maxwell Lerch

A method of implementing sub-table replication starts with the processor detecting an update to an entitlements table. The processor performs filtering of a data table based on the update to the entitlements table. The data table including an entitlements column. The processor detects an update to the entitlements column and performs incremental replication of the data table by causing a version-based replication to be executed. Other embodiments are also described herein.

System for list-based database replication

A system for data object replication includes at least one hardware processor and at least one memory storing instructions. The instructions cause the at least one hardware processor to perform operations including parsing a replication request to obtain a data object and identification information identifying a set of databases at a first deployment of a data provider. A dependency of the data object to one or more additional data objects stored in the set of databases is detected. A sequential replication of the data object and the one or more additional data object from the first deployment to a second deployment of the data provider is performed. The second deployment is identified by the replication request. A sequence of the sequential replication is based on the detected dependency.

Data object replication using database identification

A method includes generating at a source account of a database system user, a replication group object (RGO). The RGO includes a manifest identifying one or more data object types and one or more databases. A replication of data objects of the one or more data object types is performed from the one or more databases to a target account of the database system user. The replication of the data objects is performed as a unit. The RGO is replicated as a failover group object (FGO) within the target account of the database system user. A network failure is detected at the source account of the database system user. The target account of the database system user is promoted as a new source account based on the detecting of the network failure.

Failover configuration using replication group objects

Techniques for constructing transaction lock wait history showing blocker queries are described. A first transaction referencing a resource saved in a network-based data warehouse is received where the first transaction being blocked due to second transaction accessing the resource. A first telemetry event based on the first transaction being blocked is transmitted. After acquiring lock ownership of the resource by the first transaction, a second telemetry event based on acquiring lock ownership by the first transaction is transmitted.

Event driven technique for constructing transaction lock wait history

The subject technology determines a set of share objects in a replication group. The subject technology, for each object from the set of share objects, determines a set of permissions for a set of objects of databases that has been granted to each share object from the replication group. The subject technology, for each permission from the set of permissions, determines a set of dependencies, the set of dependencies comprising a second set of objects. The subject technology generates a graph structure based at least in part on the set of share objects, the set of permissions, and the set of dependencies. The subject technology traverses at least a portion of the graph structure to determine a final set of dependencies for replication. The subject technology performs a replication process based at least in part on the final set of dependencies.

Share-based replication

Techniques for creating, sharing, and using bundles (also referred to as packages) in a multi-tenant database are described herein. A bundle is a schema object with associated hidden schemas. A bundle can be created by a provider user and can be shared with a plurality of consumer users. The bundle can be used to enable code sharing and distribution without losing control while maintaining security protocols.

Bundle creation and distribution

This disclosure provides methods and techniques of data replication involving cross replication group (RG) references. The present disclosure avoids automatic replication failing when an entity in an RG refers to another entity external to the RG. The entity to be replicated within the RG is referred to as the 'referring entity,' and the entity as the dangling reference is referred to as the “referred entity.” Although the referring and referred entities are not replicated together in a replication operation, the referred entity may have already been replicated to the target account in another replication operation on a different replication group. In such cases, the data replication procedure may, according to aspects of the present disclosure, check if the referred entity has already been replicated, and if so, proceed to replicate the referring entity without fail, and link the referring and referred entities to enable normal functioning of the referring entity.

Data replication with cross replication group references

Various embodiments provide for replicating a share across deployments of a data platform, where the share can be on a source deployment and the share can be replicated on one or more target deployments, and where the share is replicated with one or more database objects of the source deployment associated with the share. Some embodiments analyze the share to be replicated and, based on the analysis, determine one or more database objects that would be replicated to the one or more target deployments to enable a replica of the share on the one or more target deployments.

Replication of share across deployments in database system

A system for sub-table replication within a materialized view of a source data table. A materialized view of a source data table is generated based on a query, the source data table having a plurality of data table rows associated with a plurality of client identifiers and including an entitlements column and data content, the entitlements column including a plurality of provider account identifiers and a plurality of consumer account identifiers. An update to an entitlements table of the source data table is detected where the entitlements table includes a plurality of entitlements table rows and where the entitlements table rows are associated with the client identifiers, the consumer account identifiers, and the provider account identifiers. An update to the source data table based on the first update to the entitlements table is detected, and an incremental replication of the materialized view of the source data table is performed.

Materialized view sub-database replication

Described herein are techniques for replicating external stages between deployments of e.g., a cloud data lake using a modified storage integration. The modified storage integration may be defined with a set of storage locations. The storage integration comprises a base URL for each of the set of storage locations and each storage location identifies a remote deployment where a cloud platform is hosted and a geographic region of the remote deployment. An external stage object may be bound to the storage integration, where the external stage object facilitates a data loading operation that is currently in progress on the first storage location. In response to detecting an outage at the first storage location, the data loading operation that was in progress on the first storage location may be continued at the second storage location using the storage integration.

Stage replication in a cloud data lake

A system for data object replication includes at least one hardware processor and at least one memory storing instructions. The instructions cause the at least one hardware processor to perform operations including decoding a replication request to obtain a data object. Object dependencies associated with the data object and a plurality of additional data objects are determined. A replication sequence of the data object and the plurality of additional data objects is determined based on the object dependencies. A replication of the data object and at least one of the plurality of additional data objects is performed according to the replication sequence. A notification of a successful completion of the replication is generated.

Data object replication using a replication sequence

System for regional database replication

Provided herein are systems and methods for configuring replication group objects. A system includes at least one hardware processor coupled to memory and configured to decode a replication request received from a client device of a data provider. The replication request indicates a plurality of account objects for replication. The plurality of account objects associated with a corresponding plurality of account object types. A replication group object is generated based on the replication request. The replication group object includes a manifest. The manifest lists the plurality of account objects. The replication of the plurality of account objects is performed from a source account of the data provider into at least one target account. The replication is based on the manifest of the replication group object.

Replication group objects configuration in a network-based database system

This disclosure provides methods and techniques of data replication involving cross replication group (RG) references. The present disclosure avoids automatic replication failing when an entity in an RG refers to another entity external to the RG. The entity to be replicated within the RG is referred to as the 'referring entity,' and the entity as the dangling reference is referred to as the 'referred entity.' Although the referring and referred entities are not replicated together in a replication operation, the referred entity may have already been replicated to the target account in another replication operation on a different replication group. In such cases, the data replication procedure may, according to aspects of the present disclosure, check if the referred entity has already been replicated, and if so, proceed to replicate the referring entity without fail, and link the referring and referred entities to enable normal functioning of the referring entity.

Growing community of inventors

Seattle, WA, United States of America

Sahaj Saini