Amazon Technologies, Inc.

Cheng Ran Li

Stefano Stefani

Rama Krishna Sandeep Pokkunuri

Ganesh Kumar Gella

Kalpesh N Sutaria

Enrico Sartorello

Sudhakar Rao Puvvadi

Nikhil Kandoi

Tania Khattar

Navneet Sabbineni

Techniques for hosting adding and warming a host are described. In some instances, a method of determining that at least one group of hosts is to be increased by adding an additional host to the group of hosts; sending a request to the group of hosts for a list of machine learning models loaded per host of the group of hosts; receiving, from each host, the list of loaded machine learning models; loading at least a proper subset of list of loaded machine learning models into random access memory of the at least one group; receiving a request to perform an inference; routing the request to the additional host of the group of hosts; performing an inference using the additional host of the group of hosts; and providing a result of the inference to an external entity is described.

Pre-warming scheme to load machine learning models

Growing community of inventors

Bellevue, WA, United States of America

Cheng Ran Li