Zhendong Wang

Title: Innovations of Zhendong Wang in Sparse Attention Modules

Introduction

Zhendong Wang is an accomplished inventor based in Plano, TX (US). He has made significant contributions to the field of technology, particularly in the development of methods and apparatuses for enhancing the efficiency of transformer models in heterogeneous devices.

Latest Patents

Zhendong Wang holds a patent titled "Method and apparatus for length-aware local tiling in a sparse attention module in a transformer." This innovative method involves dividing a transformed sparsity mask into multiple tiles and identifying effective tiles that contain non-zero elements. The process includes loading these effective tiles into shared memory on GPUs and performing matrix multiplication using a specialized kernel, thereby optimizing the performance of sparse attention modules.

Career Highlights

Zhendong Wang is currently employed at Beijing Transtreams Technology Co. Ltd., where he continues to push the boundaries of technology. His work focuses on improving the efficiency of machine learning models, particularly in the context of sparse attention mechanisms.

Collaborations

Throughout his career, Zhendong has collaborated with notable colleagues, including Yongxiong Ren and Yang Liu. These partnerships have fostered a collaborative environment that encourages innovation and the sharing of ideas.

Conclusion

Zhendong Wang's contributions to the field of technology, particularly through his patent on sparse attention modules, highlight his role as a leading inventor. His work continues to influence advancements in machine learning and artificial intelligence.