Microsoft Technology Licensing, LLC

Dharma Kiritkumar Shukla

Aditya G Bhandarkar

Karthik Raman

Akash Jeevan Sagar

Madhan Gajendran

Shireesh Kumar Thota

Abhay Vinayak Parasnis

Mayank Mehta

Robert Brian Schmidt

Pankaj Sharma

Ori Amiga

Atul Katiyar

Momin Mahmoud Al-Ghosien

Arash Ghanaie-Sichanie

John MacIntyre

Ankur Savailal Shah

Israel Hilerio

Raymond Edward Endres

Nathan Christopher Talbert

Asad Jawahar

Quetzalcoatl Bradley

Kumarswamy P Valegerepura

Junfeng Zhang

Abolade Gbadegesin

Eric Fleischman

David Richard Reed

Ji Huang

Farookh P Mohammed

Muthukaruppan Annamalai

Thomas A Galvin

Nikolay Smolyanskiy

Hari Sudan Sundar

Mikhail Mikhailovich Koltachev

Liang Li

Nagender Vedula

Sujit Vattathil Kuruvilla

Shreshth Singhal

Matthew S Augustine

Muthian Sivathanu

Roman Batoukov

Rimma V Nehme

Bhalakumaaran Erode Ranganathan

Ovidiu Constantin Platon

William R Taylor

Yann Christensen

Constantin Dulu

Mark Connolly Benvenuto

Elisa Marie Flasko

Samer Boshra

Matthew J Burdick

Nemanja Matkovic

Rajeev Sudhakar Bhopi

Subramanyam Pattipaka

Krishnan Sundaram

Li Zhang

Ramachandran Ramjee

Lu Xun

Satish Ramchandra Thatte

Bimal Kumar Mehta

Yuan Yu

Girish Mittur Venkataramanappa

David G Green

Amol Kulkarni

Arsalan Ahmad

Mihai Budiu

Nipun Kwatra

Chandra Prasad

David T Fong

Srinidhi Viswanatha

Andreas Ulbrich

Adrian Ilcu Predescu

Mandyam Kishore

Hari Krishnan

Nicholas Alexander Allen

Raymond Edward Ozzie

Stephen Harris Toub

John Anthony Taylor

Mark Eugene Russinovich

Ming Gang Liu

Jack E Ozzie

George P Moromisato

George M Moore

Raman Narayanan

Kartik Paramasivam

Perraju Bendapudi

Anil Kumar Nori

Igor Sedukhin

Balinder Malhi

Eddie Fusaro

Sergey Chub

Nishant Gupta

Sumit Mohanty

Vijayendra Gopalrao Vasu

Vaibhav Sharma

Vinod Shanbhag

Yao Lu

Hemant Kumar

Hari Krishnan S

Murali Krishnaprasad

Jayanta Mondal

The disclosure herein describes elastically managing the execution of workers of multi-worker workloads on accelerator devices. A first worker of a workload is executed on an accelerator device during a first time interval. A first context switch point is identified when the first worker is in a first worker state. At the identified context switch point, a first memory state of the first worker is stored in a host memory and the accelerator device is configured to a second memory state of the second worker. The second worker is executed during a second time interval and a second context switch point is identified at the end of the second time interval when the second worker is in a state that is equivalent to the first worker state. During the intervals, collective communication operations between the workers are accumulated and, at the second context switch point, the accumulated operations are performed.

Elastically managing workers of multi-worker workloads on accelerator devices

The disclosure herein describes scheduling execution of artificial intelligence (AI) workloads in a cloud infrastructure platform. A global scheduler receives AI workloads associated with resource ticket values. The scheduler distributes the AI workloads to nodes based on balancing resource ticket values. Local schedulers of the nodes schedule AI workloads on resources based on the resource ticket values of the AI workloads. Based on scheduling the AI workloads, coordinator services of the local schedulers execute the distributed AI workloads on the infrastructure resources of the nodes. The disclosure further describes scheduling AI workloads based on priority tiers. A scheduler receives AI workloads, and each AI workload is associated with a priority tier indicative of a preemption priority while being executed. The AI workloads are scheduled for execution on a distributed set of nodes based on the priority tiers and then execute based on the scheduling.

Scheduler for planet-scale computing system

A computing system is provided, including non-volatile storage storing a reactive database including a plurality of database entities. The computing system may further include a processor configured to, via a reactive database application program interface (API), receive a first standing query registration input including a first standing query. The first standing query may include a first update condition and may be associated with a first database entity. The processor may be further configured to store the first standing query in the non-volatile storage. The processor may be further configured to write data to the reactive database and determine that the first update condition is satisfied by the written data. In response to determining that the first update condition is satisfied, the processor may be further configured to execute the first standing query to perform a first state change at the first database entity of the plurality of database entities.

Standing query to reactive database

The disclosure herein describes platform-level checkpointing for deep learning (DL) jobs. The checkpointing is performed through capturing two kinds of state data: (i) GPU state (device state), and (ii) CPU state (host state). The GPU state includes GPU data (e.g., model parameters, optimizer state, etc.) that is located in the GPU and GPU context (e.g., the default stream in GPU, various handles created by the libraries such as DNN, Blas, etc.). Only a fraction of the GPU memory is copied because the checkpointing is done in a domain-aware manner. The 'active' memory contains useful data like model parameters. To be able to capture the useful data, memory management is controlled to identify which parts of the memory are active. Also, to restore the destination GPU to the same context/state, a mechanism is used to capture such state-changing events on an original GPU and replayed on a destination GPU.

Transparent pre-emption and migration for planet-scale computer

The disclosure herein describes platform-level migration for deep learning training (DLT) jobs from a checkpointed stated between a source node and a destination node. The checkpointing is performed through capturing GPU state (e.g., device state) and CPU state (e.g., host state). The GPU state includes GPU data (e.g., model parameters, optimizer state, etc.) that is located in the GPU and GPU context (e.g., the default stream in GPU, various handles created by libraries). Restoring the DLT job on the destination node involves resumption of processing of a destination GPU at the same checkpointed state.

Artificial intelligence workload migration for planet-scale artificial intelligence infrastructure service

The management of applications in a computing environment often involves a significant number of computing operations, such as acquiring the application, deploying data objects in various locations, and configuring the application with respect to the deployed device. Conventional computing environments may provide inadequate support throughout a typical application life cycle, especially for a computing environment distributed across many devices. Instead, the computing environment may be represented in a deployable object hierarchy that may be distributed to various devices. An application management service may therefore be provided to install representations of the application in the object hierarchy, and to support many other aspects of the application life cycle, such as application discovery, sharing, updating, and removal. The devices may therefore provide the applications of the computing environment to the user in a consistent manner with reduced manual management among the devices comprising the computing mesh.

Application management within deployable object hierarchy

A server set for a data set may designate a subset of 'master' servers that update the data set in order to reduce data version conflicts involving mutually exclusive updates of the data set. Multi-master configurations may fulfill the performance constraints, and the subset of masters may detect and resolve data version conflicts. However, if multiple masters perform conflict resolution for a particular data version conflict, the resolution may produce inefficiency and redundancy (if the masters reach the same outcome) or additional data version conflicts (if the masters reach different outcomes). Instead, among the masters, a merge master may be identified that applies conflict resolution techniques to data version conflicts and forwards the conflict resolution outcome to the other masters for application to the data set to resolve the data version conflict. The other masters may temporarily store updates in a tentative update set until data version conflicts are resolved.

Merging conflict resolution for multi-master distributed databases

Data services for workloads are often provided with a service level agreement specifying various performance guarantees (e.g., latency, availability, scalability, and consistency). Single-master architectures, in which updates to the data set are constrained to a single server, may limit the fulfillment of some performance guarantees. Presented herein are multi-master architectures, in which the server set is partitioned into at least two masters are permitted to update the data set and at least one non-master that is not permitted to update the data set. Non-masters that receive a request to update the data set forward the request to a master server for application to the data set. A master that receives the request applies it to the data set and propagates the update to other master and non-master servers. Conflicting updates may be resolved through a variety of conflict resolution techniques, optionally designating one master server as a conflict resolution server.

Multi-master architectures for distributed databases

Data services are often provided with various performance guarantees. Multi-master architectures, in which multiple master servers are permitted to update a portion of the data set, may facilitate some performance requirements, but may also lead to data version conflicts in which different masters have written different versions of one or more data items. Moreover, conflicts involving different data items may have to be resolved using different conflict resolution techniques. Therefore, various data items of the data set may be associated with a conflict resolution technique selected from a conflict resolution technique set, such as manual conflict resolution; a write order policy, such as last writer wins; a conflict resolution logic; and conflict resolution based on data types. A data version conflict may be resolved by identifying and invoking the selected conflict resolution technique that is associated with the data item, and applying the conflict resolution outcome to the data item.

Conflict resolution for multi-master distributed databases

Distributed transactions are performed over a collection of servers operating as replicas of a data set, where a successful transaction involves meeting a quorum count of replicas that locally commit the transaction. However, performance constraints of data sets and consuming applications may vary (e.g., sensitivity to latency, scalability, and/or consistency), and the performance characteristics of the server set may be partly determined by the transactional commitment and quorum selection. The distributed transaction may be applied by designating the replicas as a set of followers and a leader that initiates the transaction and receives acknowledgments of local commits by each follower. On condition of the acknowledgments meeting a quorum count for the data set according to the performance characteristics of the application, the leader locally commits the transaction and delivers a result. The transaction may also be applied over collections of replica sets using a second-level quorum to achieve nested consensus.

Resource-governed protocol and runtime for distributed databases with consistency models

An application on a device may interact with a document service that provides access to a document service. The interaction may occur in a variety of contexts, such as a device context (e.g., the document service and the application provided on the same device), a local context (e.g., via a LAN), and a remote context (e.g., over the Internet). It may be advantageous to adapt the interaction to the current context, while also providing a consistent application interface that alleviates the application from context-specific implementations, and also achieving this adaptation in a performant manner. These considerations may be achieved in a balanced manner by providing a set of runtimes, where each runtime mediates the application/document service interaction between the application and the document service in a contextually adapted manner. The device may automatically detect the context of the interaction and select a contextually suitable runtime to service the application.

Available, scalable, and tunable document-oriented storage services

Data sets such as databases are often distributed over a number of servers, where each server stores a subset of records and an index that enables the server to locate the records in response to queries. However, tight coupling of indexing and storage may limit load-balancing, fault recovery, and distribution. Instead, the set of servers may be partitioned into a set of storage servers that store the records and a set of index servers of the index over the records. In a set that is decoupled in this manner, load-balancing may involve provisioning and locating index servers independently of the provisioning and locating of the storage servers in view of the particular sensitivities and tolerances of various applications. Additionally, the index servers may also utilize a data layout that is selected and adapted independent of the data layout of the storage servers and/or the schema of the data stored thereby.

Schema-agnostic indexing of distributed databases

Growing community of inventors

Bellevue, WA, United States of America

Dharma Kiritkumar Shukla