Tata Consultancy Services Limited

Ranjitha Prasad

Gautam Shroff

Lovekesh Vig

Garima Gupta

Ramya Hebbalaguppe

Ankit Sharma

Srinidhi Hegde

Arnab Chatterjee

Srinidhi Hedge

Implementations of the present disclosure build a Bayesian student network using the knowledge learnt by an accurate but complex pre-trained teacher network, and sparsity induced by the variational parameters in a student network. Further, the sparsity inducing capability of the teacher on the student network is learnt by employing a Block Sparse Regularizer on a concatenated tensor of teacher and student network weights. Specifically, the student network is trained using the variational lower bound based loss function, constrained on the hint from the teacher, and block-sparsity of weights.

Sparsity constraints and knowledge distillation based learning of sparser and compressed neural networks

Growing community of inventors

Gurgaon, India

Ranjitha Prasad