The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Jul. 29, 2025
Filed:
Jun. 12, 2023
Microsoft Technology Licensing, Llc, Redmond, WA (US);
Marko Radmilac, Bellevue, WA (US);
Andrew James Wald, Redmond, WA (US);
Joshua Bryan Wyman Clemons, Seattle, WA (US);
Microsoft Technology Licensing, LLC, Redmond, WA (US);
Abstract
A model comprises different processing nodes that perform different functions on input data, such as data prompts. Each node has batching criteria including a threshold value that when exceeded triggers a transmission of a batch of processing requests to the corresponding node. Different processing requests for a node are routed to a batching cache for that node. Processing requests are prevented from being routed to the node until the batching criteria is met. Once the batching criteria is met, a batch of processing requests stored in the batching cache are routed to the corresponding node.