Microsoft Technology Licensing, LLC

Nikhil Devanur Rangarajan

Amar Phanishayee

Timothy S Paek

Kamal Jain

Bongshin Lee

Dmytro Rudchenko

Keqing Liang

Wen Pu

Jorgen Thelin

Debmalya Panigrahi

Giorgio Paolo Martini

Vivek Seshadri

Christopher A Wilkens

Shivaram Venkataraman

Guanhua Wang

Aaron Harlap

David Pennock

Deepak Narayanan

Jennifer Wortman Vaughan

Yiling Chen

Zhiyi Huang

Balasubramanian Sivan

Layers of a deep neural network (DNN) are partitioned into stages using a profile of the DNN. Each of the stages includes one or more of the layers of the DNN. The partitioning of the layers of the DNN into stages is optimized in various ways including optimizing the partitioning to minimize training time, to minimize data communication between worker computing devices used to train the DNN, or to ensure that the worker computing devices perform an approximately equal amount of the processing for training the DNN. The stages are assigned to the worker computing devices. The worker computing devices process batches of training data using a scheduling policy that causes the workers to alternate between forward processing of the batches of the DNN training data and backward processing of the batches of the DNN training data. The stages can be configured for model parallel processing or data parallel processing.

Highly performant pipeline parallel deep neural network training

An interconnect topology for communication between GPUs in a computing system is determined. A quantity of directed spanning trees are generated for transmitting data between the GPUs using the interconnect topology and packed. The directed spanning trees define the connections between GPUs that are to be utilized for the transmission and the amount of data to be transmitted on each connection. Program code is generated for implementing the data transfer defined by the directed spanning trees. When the program code is executed, the directed spanning trees are used to pipeline the transmission of chunks of data, such as model parameters used during data-parallel DNN training, between the GPUs. The program code can also determine an optimal chunk size for data to be transferred between the GPUs.

Mitigating communication bottlenecks during parameter exchange in data-parallel DNN training

Techniques for the dynamic slotting of content items within electronic content are provided. In response to receiving a request, a first set of content items is identified and a first score is generated for each based on a first objective. Based on the first scores, a first ranking of the first set of content items is generated. A subset of the first set is selected based on the first ranking. A second set of content items that includes the subset is identified and a second score is generated for each item in the second set based on a second objective that is different than the first objective. Based on the second scores, a second ranking of the second set is generated. A subset of the second set is selected based on the second ranking. The second subset is presented on a computing device that is associated with the request.

Dynamic slotting of content items within electronic content

A computer system includes a server computer that handles transactions with user devices to create and manage events, create and manage user accounts and process wagers. Transactions regarding wagers in connection with events can be processed in real-time. The wagers are processed using a payoff function that rewards participation and accurate information, yet does not permit arbitrage or otherwise reward splitting a wager by participating under multiple identities. To compute a payoff, a first score is computed based on the probability estimate by a participant and the outcome. A second score is computed based on an aggregate of wagers of all other participants for each participant, where the first score for the participant is no more than the second score for at least one outcome. The payoff for the participant is then computed as a function of the first score and the second score.

Computer system for multiple user, multiple event real-time online wagering

The subject disclosure is directed towards scheduling jobs with a speed for running a processor(s) having variable speeds to save energy yet complete in time, in which the volume of the job is not known in advance, that is, in a non-clairvoyant setting. A non-clairvoyant algorithm uses an existing clairvoyant algorithm to determine the speed based upon information known from running one or more jobs, in full or in part. Also described is rounding jobs based upon their densities into rounding queues so that a hybrid of highest density first rules and FIFO rules may be used to obtain information used by the clairvoyant algorithm.

Energy efficient job scheduling

A soft input panel (SIP) for a computing device is configured to be used by a person holding a computing device with one hand. For example, a user grips a mobile computing device with his right hand at the bottom right corner and uses his right thumb to touch the various keys of the SIP, or grips a mobile computing device with his left hand at the bottom left corner and uses his left thumb to touch the various keys of the SIP. The SIP comprises arced or slanted rows of keys that correspond to the natural pivoting motion of the user's thumb.

Arced or slanted soft input panels

Various embodiments provide offline algorithms for resource allocation. A known set of 'offline' requests may be matched to available resources using an online resource allocation algorithm that models the offline resource allocation problem as though the requests were received stochastically. Requests may be scaled and then sampled to provide random, stochastic input for the online resource allocation algorithm. For each request, resources are allocated to the request by evaluating multiple options based upon shadow costs assigned to resources associated with the different options. After each request is processed, an adjustment is made to the shadow costs for remaining resources to reflect differences in rates for allocation and/or consumption of the resources and the updated shadow costs are used for a subsequent request. A scaled resource allocation determined using sampled requests in this manner may be scaled back up to obtain a solution for the offline resource allocation problem.

Offline resource allocation algorithms

Growing community of inventors

Seattle, WA, United States of America

Nikhil Devanur Rangarajan