Beijing Transtreams Technology Co. Ltd.

Xing Wen

Yang Liu

Lingzhi Liu

Yongxiong Ren

Methods and apparatuses are provided for serving multiple models with a single engine. The method includes: providing a single engine for a plurality of models on a server, where the server includes at least one graphics processing unit (GPU) and memory coupled with the at least one GPU; loading the plurality of models onto the memory of the server at once; and serving, by the single engine, the plurality of models, where the single engine accommodates all structures and weights of the plurality of models with a shared input and output of the memory.

Growing community of inventors

Palo Alto, CA, United States of America

Xing Wen