Qpiai India Private Limited

Aswanth Krishnan

Nagendra Nagaraja

Sachin Kumar

Lakshya Priyadarshi

Systems and methods for optimizing the placement of experts in a mixture-of-experts (MoE) model are disclosed. A system can include one or more processors coupled to non-transitory memory. The system can obtain a set of attributes of a plurality of hardware components of a distributed computing environment. The system can determine a set of co-activation metrics for a plurality of experts of a mixture-of-experts model according to inference or training operations executed using the mixture-of-experts model. The system can generate a quadratic unconstrained binary optimization (QUBO) data structure based at least on the set of attributes and the set of co-activation metrics. The system can assign the plurality of experts of the mixture-of-experts model to the plurality of hardware components of the distributed computing environment according to the QUBO data structure. The system can execute the mixture-of-experts model using the plurality of hardware components according to the assignment.

Growing community of inventors

Bengaluru, India

Aswanth Krishnan