Microsoft Technology Licensing, LLC

Marko Radmilac

Genevieve Fernandes

Paul F Ringseth

William Robert Messmer

Niklas Gustafsson

Andrew James Wald

Joshua Bryan Wyman Clemons

Krishnan Varadarajan

Donald James McCrady

Louis Lafreniere

Michael L Chu

Charles L Mitchell

Andrew Edward Ayers

Christopher Martin McKinsey

Michael Stall

Krishan Varadarajan

A model comprises different processing nodes that perform different functions on input data, such as data prompts. Each node has batching criteria including a threshold value that when exceeded triggers a transmission of a batch of processing requests to the corresponding node. Different processing requests for a node are routed to a batching cache for that node. Processing requests are prevented from being routed to the node until the batching criteria is met. Once the batching criteria is met, a batch of processing requests stored in the batching cache are routed to the corresponding node.

Dynamic batching of prompt queries

A meta-model topology comprises a plurality of functions and conforms to a global label schema. A new function not included in the plurality of functions is integrated into the meta-model topology. A particular label of interest that is associated with the new function is identified and the new function is configured such that an output from the new function conforms to an output form corresponding to the particular label of interest from the global label schema. The new function is then integrated into the meta-model topology and the meta-model topology that includes the new function is used to generate a model graph. The model graph is then deployed to a remote application that is configured to receive data prompts comprising input data processed by nodes of the model graph.

Deployment of meta-model topologies and moderating graph generation

A concurrent data structure allows synchronization to be elided for read accesses. Processing resources that remove one or more elements of the concurrent data structure are allowed to delete the elements only after all other processing resources have reached a safe point. Each processing resource maintains an indicator that indicates whether the processing resource has reached as safe point (i.e., will not access the concurrent data structure). When the indicators indicate that all processing resources have reached a safe point, elements of the data structure may be deleted.

Eliding synchronization in a concurrent data structure

An interface between a resource manager and schedulers in a process executing on a computer system allows the resource manager to manage the resources of the schedulers. The resource manager communicates with the schedulers using the interface to access statistical information from the schedulers. The statistical information describes the amount of use of the resources by the schedulers. The resource manager also communicates with the schedulers to dynamically allocate and reallocate resources among the schedulers in the same or different processes or computer systems in accordance with the statistical information.

Interface between a resource manager and a scheduler in a process

Each processing resource in a scheduler of a process executing on a computer system maintains counts of the number of tasks that arrive at the processing resource and the number of tasks that complete on the processing resource. The counts are maintained in storage that is only writeable by the corresponding processing resource. The scheduler collects and sums the counts from each processing resource and provides statistics based on the summed counts and previous summed counts to a resource manager in response to a request from the resource manager. The scheduler does not reset the counts when the counts are collected and stores copies of the summed counts for use with the next request from the resource manager. The counts may be maintained without synchronization and with thread safety to minimize the impact of gathering statistics on the application.

Monitoring and updating tasks arrival and completion statistics without data locking synchronization

Systems, methods, and computer program products for facilitating a serial debugging experience for programs running in a distributed, non-deterministic, out-of-order (i.e., parallel) fashion are disclosed. Such systems, methods, and computer program products provide a modified combination of breakpoints, stepping, and call stacks that facilitate the ability to improve the debugging experience for out-of-order program execution. In an embodiment, a task scheduler provides a distributed task graph. Then, the user may hit breakpoints sequentially within the graph's execution and use deterministic stepping (i.e., step-in, step-out, and step-over functions) to create a logical call stack from a distributed task graph. Such embodiments allow the program to execute in a distributed and parallel manner, while presenting the user a sequential execution experience in the debugger.

Serial debugging experience for parallel programs

A resource manager manages processing and other resources of schedulers of one or more processes executing on one or more computer systems. For each scheduler, the resource manager determines an initial allocation of resources based on the policy of the scheduler, the availability of resources, and the policies of other schedulers. The resource manager receives feedback from the schedulers and dynamically changes the allocation of resources of schedulers based on the feedback. The resource manager determines if changes improved the performance of schedulers and commits or rolls back the changes based on the determination.

Dynamically change allocation of resources to schedulers based on feedback and policies from the schedulers and availability of the resources

Embodiments are directed to dynamically allocating processing resources among a plurality of resource schedulers. A resource manager dynamically allocates resources to a first resource scheduler. The resource manager is configured to dynamically allocate resources among a plurality of resource schedulers, and each scheduler is configured to manage various processing resources. The resource manager determines that at least one of the processing resources dynamically allocated to the first resource scheduler is idle. The resource manager determines that at least one other resource scheduler needs additional processing resources and, based on the determination, loans the determined idle processing resource of the first resource scheduler to a second resource scheduler.

Efficient utilization of idle resources in a resource manager

Annotation of exception information in a computer program. This annotation is not present in the source code itself, but is annotated during the compilation process or perhaps for purpose of analyzing the code. To perform the annotation, environmental rules are accessed regarding the environment in which the program is to be executed. The program is then evaluated in the context of these environmental rules to detect portions that, in the context of the environmental rules, may cause exceptions to be thrown. Those portions may then be annotated with exception information that describes any exceptions that could be thrown at that detected portion.

Growing community of inventors

Bellevue, WA, United States of America

Marko Radmilac