The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Jan. 10, 2023
Filed:
Mar. 05, 2021
Microsoft Technology Licensing, Llc, Redmond, WA (US);
Hao Huang, Kenmore, WA (US);
Zhenghua Yang, Sammamish, WA (US);
Long Qiu, Kirkland, WA (US);
Ashish Pinninti, Redmond, WA (US);
Juan Diego Ferre, Seattle, WA (US);
Amit Anand Amleshwaram, Wynnewood, PA (US);
Microsoft Technology Licensing, LLC, Redmond, WA (US);
Abstract
Techniques for machine learning inferencing endpoint discovery in a distributed computing system are discloses herein. In one example, a method includes searching a database containing machine learning endpoint records having data representing values of execution latency or prediction accuracy corresponding inferencing endpoints deployed in the distributed computing system. The method also includes generating a list of inferencing endpoints matching the individual target values and determining whether a count of the inferencing endpoints in the generated list exceeds a preset threshold. In response to determining that the identified count does not exceed the preset threshold, the method includes instantiating one or more additional inferencing endpoints in the distributed computing system based on the individual target values in the received query.