Baidu USA LLC

Baidu.com Times Technology (beijing) Co. Ltd.

Ruichang Cheng

Lei Wang

Manjiang Zhang

Zhen Chen

Davy Huang

Xing Li

Haofeng Kou

Huimeng Zheng

A system determines a timing matrix corresponding to inference times taken for a number of machine learning (ML) models to be executed by a number of processing resources of a computing device. The processing resources includes at least a first and a second type of processing resources. The system applies a service-specific model-first scheduling scheme or a service-specific hardware-first scheduling scheme to obtain corresponding service-specific mappings. The system determines a best mapping from the corresponding service-specific mappings. The system schedules each of the ML models to a corresponding processing resource from the processing resources according to the best mapping. The system executes the ML models using corresponding mapped processing resources.

Scheduling ML services and models with heterogeneous resources

A system obtains a performance profile corresponding to times taken to perform an inferencing by a machine learning (ML) model using a different number of processing resources from a plurality of processing resources. The system determines one or more groupings of processing resources from the plurality of processing resources, each grouping includes one or more partitions. The system calculates performance speeds corresponding to each grouping based on the performance profile. The system determines a grouping having a best performance speed from the calculated performance speeds. The system partitions the processing resources based on the determined grouping to perform the inferencing.

System on chip parallel computing of ML services and applications for partitioning resources based on the inference times

Growing community of inventors

Beijing, China

Ruichang Cheng