Cong Wei

Title: Cong Wei: Innovator in Vision Transformers

Introduction

Cong Wei is a notable inventor based in Toronto, Canada. He has made significant contributions to the field of artificial intelligence, particularly in the development of efficient vision transformers. His innovative work focuses on enhancing the performance of these models while reducing their computational costs.

Latest Patents

Cong Wei holds a patent for "Systems, methods and techniques for learning and using sparse instance-dependent attention for efficient vision transformers." This patent addresses the high computational costs associated with vision transformers (ViT) compared to convolutional neural networks (CNNs). The methods and systems he developed learn instance-dependent attention patterns, utilizing a lightweight connectivity predictor module to estimate connectivity scores between tokens. This approach allows for sparse computations, significantly accelerating the network's performance. The resulting sparse attention ViT achieves a superior balance between floating point operations (FLOPs) and top-1 accuracy on ImageNet, demonstrating a reduction of over 60% in FLOPs while maintaining a minimal accuracy drop.

Career Highlights

Cong Wei is currently employed at L'Oréal, where he applies his expertise in artificial intelligence and machine learning. His work has positioned him as a key player in the advancement of technology within the company.

Collaborations

He collaborates with talented coworkers such as Brendan Duke and Ruowei Jiang, contributing to a dynamic and innovative work environment.

Conclusion

Cong Wei's contributions to the field of vision transformers exemplify the intersection of technology and efficiency. His innovative patent and work at L'Oréal highlight his commitment to advancing artificial intelligence.