Microsoft Technology Licensing, LLC

Pradeep Kumar Gunda

Bradley Gene Calder

Ju Wang

Sriram Sankaran

Hemal Khatri

Yang Yang Zhang

Marvin McNett, Ii

Shyam Antony

Kavitha Manivannan

Vaman Bedekar

Arild Einar Skjolsvold

Yuan Yu

Michael Isard

Li Zhuang

Vanam Bedekar

Marvin Mcnett Ii

Yang Zhang

Systems and methods are provided for assigning and associating resources in a cloud computing environment. Virtual machines in the cloud computing environment can be assigned or associated with pools corresponding to users as dedicated, standby, or preemptible machines. The various states provide users with the ability to reserve a desired level of resources while also allowing the operator of the cloud computing environment to increase resource utilization.

Assignment of resources in virtual machine pools

In various embodiments, systems and methods are presented for providing resources by way of a platform as a service in a distributed computing environment to perform a job. The system may be comprised of a number of components, such as a task machine, a task location service machine, and a high-level location service machines that in combination are useable to accomplish functions provided herein. It is contemplated that the system performs methods for providing resources by determining resources of the system, such as virtual machines, and applying auto-scaling rules to the system to scale those resources. Based on the determination of the auto-scaling rules, the resources may be allocated to achieve a desired result.

Auto-scaling of pool of virtual machines based on auto-scaling rules of user associated with the pool

Systems and methods are presented for providing resources by way of a platform as a service in a distributed computing environment to perform a job. Resources of the system, job performing on the system, and schedulers of the jobs performing on the system are decoupled in a manner that allows a job to easily migrate among resources. It is contemplated that the migration of jobs from a first pool of resource to a second pool of resource is performed by the system without human intervention. The migration of a job may utilize different schedulers for the different resources. Further, it is contemplated that a pool of resources may automatically allocate additional or fewer resources in response to a migration of a job.

Migration of task to different pool of resources based on task retry count during task lease

Data and computation may be unified (i.e., integrated) in a data center using a single query interface. Users may interact with the data center via a query interface to provide a query (i.e., a computation) to the data center. The results of the query may be referred to as derived datasets and may be managed by a cache server. In an implementation, a derived dataset is uniquely referenced by the query that computes it. Shared common computations are computed only once and may be reused by other computations. The result of a query may be computed (if not previously cached) and returned to the user. Infrequently used derived datasets may be garbage collected (e.g., deleted or otherwise removed from storage) by a garbage collector. This integration of data and computation provides efficient resource management for data center.

Management of data and computation in data centers

General-purpose distributed data-parallel computing using high-level computing languages is described. Data parallel portions of a sequential program written in a high-level language are automatically translated into a distributed execution plan. Map and reduction computations are automatically added to the plan. Patterns in the sequential program can be automatically identified to trigger map and reduction processing. Direct invocation of map and reduction processing is also provided. One or more portions of the reduce computation are pushed to the map stage and dynamic aggregation is inserted when possible. The system automatically identifies opportunities for partial reductions and aggregation, but also provides a set of extensions in a high-level computing language for the generation and optimization of the distributed execution plan. The extensions include annotations to declare functions suitable for these optimizations.

General distributed reduction for data parallel computing

Growing community of inventors

Bellevue, WA, United States of America

Pradeep Kumar Gunda