Microsoft Technology Licensing, LLC

Rimma V Nehme

Dharma Kiritkumar Shukla

Muthian Sivathanu

Srinath Shankar

Ramachandran Ramjee

Nicolas Bruno

Lu Xun

Pankaj Sharma

Nipun Kwatra

Srinidhi Viswanatha

Alan D Halverson

Atul Katiyar

Bhalakumaaran Erode Ranganathan

Vaibhav Sharma

Shreshth Singhal

Eric R Robinson

Bhargav Gulavani

Ravi Shreyas Anupindi

Amey Agrawal

Kaustubh Welankar

The disclosure herein describes elastically managing the execution of workers of multi-worker workloads on accelerator devices. A first worker of a workload is executed on an accelerator device during a first time interval. A first context switch point is identified when the first worker is in a first worker state. At the identified context switch point, a first memory state of the first worker is stored in a host memory and the accelerator device is configured to a second memory state of the second worker. The second worker is executed during a second time interval and a second context switch point is identified at the end of the second time interval when the second worker is in a state that is equivalent to the first worker state. During the intervals, collective communication operations between the workers are accumulated and, at the second context switch point, the accumulated operations are performed.

Elastically managing workers of multi-worker workloads on accelerator devices

The disclosure herein describes scheduling execution of artificial intelligence (AI) workloads in a cloud infrastructure platform. A global scheduler receives AI workloads associated with resource ticket values. The scheduler distributes the AI workloads to nodes based on balancing resource ticket values. Local schedulers of the nodes schedule AI workloads on resources based on the resource ticket values of the AI workloads. Based on scheduling the AI workloads, coordinator services of the local schedulers execute the distributed AI workloads on the infrastructure resources of the nodes. The disclosure further describes scheduling AI workloads based on priority tiers. A scheduler receives AI workloads, and each AI workload is associated with a priority tier indicative of a preemption priority while being executed. The AI workloads are scheduled for execution on a distributed set of nodes based on the priority tiers and then execute based on the scheduling.

Scheduler for planet-scale computing system

The disclosure herein describes platform-level checkpointing for deep learning (DL) jobs. The checkpointing is performed through capturing two kinds of state data: (i) GPU state (device state), and (ii) CPU state (host state). The GPU state includes GPU data (e.g., model parameters, optimizer state, etc.) that is located in the GPU and GPU context (e.g., the default stream in GPU, various handles created by the libraries such as DNN, Blas, etc.). Only a fraction of the GPU memory is copied because the checkpointing is done in a domain-aware manner. The 'active' memory contains useful data like model parameters. To be able to capture the useful data, memory management is controlled to identify which parts of the memory are active. Also, to restore the destination GPU to the same context/state, a mechanism is used to capture such state-changing events on an original GPU and replayed on a destination GPU.

Transparent pre-emption and migration for planet-scale computer

The disclosure herein describes platform-level migration for deep learning training (DLT) jobs from a checkpointed stated between a source node and a destination node. The checkpointing is performed through capturing GPU state (e.g., device state) and CPU state (e.g., host state). The GPU state includes GPU data (e.g., model parameters, optimizer state, etc.) that is located in the GPU and GPU context (e.g., the default stream in GPU, various handles created by libraries). Restoring the DLT job on the destination node involves resumption of processing of a destination GPU at the same checkpointed state.

Artificial intelligence workload migration for planet-scale artificial intelligence infrastructure service

The present invention extends to methods, systems, and computer program products for optimizing parallel queries using interesting distributions. For each logical operator in an SQL server MEMO, in a top down manner from a root operator to the leaf operators, interesting distributions for the operators can be identified based on the properties of the operators. Identified interesting distributions can be propagated down to lower operators by annotating the lower operators with the interesting distributions. Thus, a SQL server MEMO can be annotated with interesting distributions propagated top down from root to leaf logical operators to generate an annotated SQL server MEMO. Parallel query plans can then be generated from the annotated SQL server MEMO in a bottom up manner from leaf operators to a root operator. Annotated interesting properties can be used to prune operators, thereby facilitating a more tractable search space for a parallel query plan.

Optimizing parallel queries using interesting distributions

Presented is a system and method for evaluating relational database queries in a distributed system. An optimized query plan is received by a control node. The query plan is decoded to a SQL statement that is semantically equivalent to the query plan, even though the query plan contains elements that have no direct analog in SQL. The decoded SQL is transmitted to a compute node for execution.

SQL generation for assert, update and delete relational trees

A single system image is provided for a parallel data warehouse system by exposing a shell database within a database management system comprising metadata and statistics regarding externally stored data. Further, functionality of the database management system can be exploited to perform pre-execution tasks. In one instance, one or more execution plans can be generated by the database management system for an input command and subsequently employed to generate a distributed execution plan.

Single system image via shell database

The min-repro finding technique described herein is designed to ease and speed-up the task of finding a min-repro, a minimum configuration that reproduces a problem in database-related products. Specifically, in one embodiment, the technique simplifies transformations in order to find one or more min-repros. One embodiment provides a high-level script language to automate some sub-tasks and to guide the search for a simpler the configuration that reproduces the problem. Yet another embodiment provides record-and-replay functionality, and provides an intuitive representation of results and the search space. These tools can save hours of time for both customers and testers to isolate the problem and can result in faster fixes and large cost savings to organizations.

Min-repro framework for database systems

Embodiments are directed to determining optimal partition configurations for distributed database data and to implementing parallel query optimization memo data structure to improve partition configuration cost estimation efficiency. In an embodiment, a computer system accesses a portion of database data and various database queries for a given database. The computer system determines, based on the accessed database data and database queries, a partition configuration search space which includes multiple feasible partition configurations for the database data and a workload of queries expected to be executed on that data. The computer system performs a branch and bound search in the partition configuration search space to determine which data partitioning path has the lowest partitioning cost. The branch and bound search is performed according to branch and bound search policies. The computer system also outputs the partition configuration with the determined lowest partitioning cost.

Automated partitioning in parallel database systems

Described herein are techniques for Configuration-Parametric Query Optimization (C-PQO) that can improve performance of database tuning tools. When first optimizing a query, a compact representation of the optimization space is generated. The representation can then be used to efficiently produce other execution plans for the query under arbitrary hypothetical configurations.

Growing community of inventors

Madison, WI, United States of America

Rimma V Nehme