Tata Consultancy Services Limited

Ashwin Krishnan

Manoj Karunakaran Nambiar

Rekha Singhal

High-performance deployment of DNN recommendation models heavily rely on embedding tables, and their performance bottleneck lies in the latency of embedding access. To optimize the deployment of RMs, the method and system is disclosed, which leverages heterogeneous memory types on FPGAs to improve the overall performance by maximizing the availability of frequently accessed data in faster memory. The system, using a optimizer dynamically allocates table partitions of the embedding tables based on history of input access history. A pre-optimizer block disclosed determines whether smaller tables should be partitioned or placed entirely in smaller memories, improving overall efficiency. The performance of RM is improved with improvement in average embedding fetch latency and effectively inference latency via modified Round Trip computation.

Growing community of inventors

Thane, India

Ashwin Krishnan