Microsoft Technology Licensing, LLC

Joshua Bryan Wyman Clemons

Marko Radmilac

Andrew James Wald

A model comprises different processing nodes that perform different functions on input data, such as data prompts. Each node has batching criteria including a threshold value that when exceeded triggers a transmission of a batch of processing requests to the corresponding node. Different processing requests for a node are routed to a batching cache for that node. Processing requests are prevented from being routed to the node until the batching criteria is met. Once the batching criteria is met, a batch of processing requests stored in the batching cache are routed to the corresponding node.

Dynamic batching of prompt queries

A meta-model topology comprises a plurality of functions and conforms to a global label schema. A new function not included in the plurality of functions is integrated into the meta-model topology. A particular label of interest that is associated with the new function is identified and the new function is configured such that an output from the new function conforms to an output form corresponding to the particular label of interest from the global label schema. The new function is then integrated into the meta-model topology and the meta-model topology that includes the new function is used to generate a model graph. The model graph is then deployed to a remote application that is configured to receive data prompts comprising input data processed by nodes of the model graph.

Growing community of inventors

Seattle, WA, United States of America

Joshua Bryan Wyman Clemons