Huawei Technologies Co., Limited

Snowflake Inc.

Huawei Cloud Computing Technologies Co., Ltd.

Xiaodi Ke

Subramanian Muralidhar

Damien Carru

Unmesh Jagtap

Lei Guo

Chong Chen

Pui Kei Johnston Chu

Jason T S Lam

Tyson J Hamilton

Xie Xu

Steven Parkes

James Pan

Haroldo Level

James G Pan

Zhenhua Hu

Chen Chen

Jin Chen

Zhi Zhao

Zhigang Luo

Khaled Ziya Aeen

Shane Anthony Bergsma

Jason TS Lam

Disclosed is an execution information sharing system that writes execution information to a provider target (and other targets) in a secure manner. Execution information generated by an application may be written to a consumer stage, wherein the application is shared by a provider account of a data exchange with a consumer account that executes the application. A consumer exchange service (ES) of the data exchange may send a request to a copy service of the data exchange to copy the execution information from the consumer stage to the provider stage, wherein the consumer ES is a part of the data exchange and is protected from actions of the consumer account. A copy operation may be executed to copy the execution information from the consumer stage to the provider stage using the copy service of the data exchange. The execution information is ingested from the provider stage to a provider table.

Sharing events and other metrics in native applications

Embodiments of the present disclosure provide a region-specific events account that is used as a central place to store the events shared by consumers of shared applications in that region. For each of a set of provider accounts in a first region, a global account object corresponding to the provider account is replicated to a second region to establish a replicated global account object in the second region. An account data persistence object (DPO) of a first provider account of the set of provider accounts is modified to designate the first provider account as an events account. The events account is a central location within the second region where execution information generated by execution of applications shared with consumer accounts in the second region is stored. The replicated global account object corresponding to the first provider account is updated to designate the first provider account as the events account.

Events account for native app event sharing

Disclosed is an execution information sharing system that duplicates execution information to a provider target (and other targets) as it is being loaded to a consumer target. A first log information object and a second log information object are generated. The first and second log information objects comprise information indicating a consumer target and information indicating a provider target respectively where execution information generated by an application shared with a consumer account of a data exchange is written. A first event unloader and a second event unloader are generated based on the first and second log information objects respectively, wherein the first and second event unloaders are both linked to the application using a mapping. In response to receiving execution information from the application, the execution information is forwarded to the consumer target and the provider target using the first event unloader and the second event unloader respectively.

Embodiments of the present disclosure provide a region-specific events account that is used as a central place to store the events shared by consumers of shared applications in that region. Use of such an account helps reduce the operational burden of a provider of the applications as they do not need to determine which accounts have shared events for each different consumer. The provider only needs to login to the events account which is in the same region as the consumer and can query all shared events from different applications that are being used in that region. Also, since the provider can designate an events account in each region, the shared events could be directly ingested into the events account without traveling to a different region.

There is provided a method and apparatus for vectorized resource scheduling using multi-dimensional hierarchical arrays with a uniform data type. According to embodiments, the hierarchy of resources may be represented as a multi-dimensional array of a uniform data type in a scheduling context. This data structure is known as a tensor. In various embodiments, a tensor may be used to model each of resource usage, requested resources, resource capabilities, tenant resource usage, etc. There is provided tensor scheduling operations (e.g. tensor scheduling algorithms) to make use of this resource model. Some embodiments may exploit specialized hardware, such as GPUs and tensor-processing units, in order to accelerate the tensor scheduling operations.

Method and apparatus for vectorized resource scheduling in distributed computing systems using tensors

Disclosed is an execution information sharing system that duplicates execution information to a provider target (and other targets) as it is being loaded to a consumer target. A consumer account of a data sharing platform executes an application shared with it by a provider account of the platform. Consumer and provider configurations indicating consumer and provider targets respectively are generated. The consumer configuration and provider configurations are provided to an event context to generate a first and second event unloaders respectively, wherein the event context maintains a mapping linking both the first event unloader and the second event unloader to the application. In response to receiving execution information from the application, the first event unloader and the second event unloader are retrieved. The execution information is then written to the consumer target and the provider target using the first event unloader and the second event unloader respectively.

A method for managing file system replication can include: collecting event data associated with file system events, each of the file system events indicative of a modification of at least one file at a first file system location; for each of the file system events, generating at least one file system action to apply the modification of the at least one file at the first file system location to at least one corresponding file at a second file system location; storing the at least one generated file system action in an action buffer; identifying dependencies between file system actions in the action buffer; and scheduling the file system actions in the action buffer for execution based on the dependencies.

Systems, devices and methods for managing file system replication

Systems and methods are disclosed herein for allocating computing resources for execution of workloads in a distributed computing system with resource acquisition cycles and workload scheduling cycles. One or more resource offer managers transmit resource offers to each of one or more of schedulers. Each scheduler transmits a resource request and/or resource release request in response to each resource offer manager. Each resource offer manager computes a resource allocation for each scheduler after resolving conflicts and assigns the resources to each scheduler according to the resource allocation decisions. The resource acquisition cycle and the workload scheduling cycle are decoupled.

Systems and methods for allocating computing resources in distributed computing

In a distributed computing network, requests for allocation of resources to tenant workloads and messages identifying resource availability are received and aggregated. Resources are allocated to the workloads in accordance with a distribution policy defining values for resource entitlements of the tenants. The values include pre-emption quantities. In response to determining that a quantity of resources allocated for workloads of a first tenant is less than the tenant's pre-emption quantity, processing of another workload from a second tenant is interrupted to re-allocate resources from the second tenant's workload to the first tenant's workload.

Method and system for multi-tenant resource distribution

Methods and systems of managing a resource in a distributed resource management system can include: receiving a resource request including data identifying at least one requested resource attribute; identifying one or more candidate resources from a plurality of resources by performing element-wise operations on one or more resource attribute element sets corresponding to the at least one requested resource attribute, each resource attribute element set comprising an element for each of the plurality of resources, each element representing a resource attribute for a respective resource of the plurality of resources; and scheduling a workload associated with the resource request for execution based on the one or more identified candidate resources.

System and method for distributed resource management

Methods and systems of managing workloads and resources for tenants structured as a hierarchical tenant queue defining a resource distribution policy. The tenants are modified and the system responds with dynamic adjustments to the hierarchical tenant queue and workload distribution. A rule-based workload management engine defines rules for actions for the workload to respond to the dynamic update.

Growing community of inventors

Markham, Canada

Xiaodi Ke