The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Sep. 06, 2022
Filed:
Sep. 28, 2018
Amazon Technologies, Inc., Seattle, WA (US);
Nikhil Kandoi, Seattle, WA (US);
Ganesh Kumar Gella, Redmond, WA (US);
Rama Krishna Sandeep Pokkunuri, Redmond, WA (US);
Sudhakar Rao Puvvadi, Bellevue, WA (US);
Stefano Stefani, Issaquah, WA (US);
Kalpesh N. Sutaria, Seattle, WA (US);
Enrico Sartorello, Berlin, DE;
Tania Khattar, Seattle, WA (US);
Amazon Technologies, Inc., Seattle, WA (US);
Abstract
Techniques for hosting machine learning models are described. In some instances, a method of receiving a request to perform an inference using a particular machine learning model; determining a group of hosts to route the request to, the group of hosts to host a plurality of machine learning models including the particular machine learning model; determining a path to the determined group of hosts; determining a particular host of the group of hosts to perform an analysis of the request based on the determined path, the particular host having the particular machine learning model in memory; routing the request to the particular host of the group of hosts; performing inference on the request using the particular host; and providing a result of the inference to a requester is performed.