Baidu USA LLC

Dario Amodei

Bryan Catanzaro

Jesse Engel

Erich Elsen

Shubhabrata Sengupta

Awni Hannun

Xu Han

Ryan Prenger

Jun Zhan

Sanjeev Satheesh

Zhenyao Zhu

Chong Wang

Christopher Fougner

Mike Chrzanowski

Jingdong Chen

Dani Yogatama

Gregory Diamos

Systems and methods for a multi-core optimized Recurrent Neural Network (RNN) architecture are disclosed. The various architectures affect communication and synchronization operations according to the Multi-Bulk-Synchronous-Parallel (MBSP) model for a given processor. The resulting family of network architectures, referred to as MBSP-RNNs, perform similarly to a conventional RNNs having the same number of parameters, but are substantially more efficient when mapped onto a modern general purpose processor. Due to the large gain in computational efficiency, for a fixed computational budget, MBSP-RNNs outperform RNNs at applications such as end-to-end speech recognition.

Systems and methods for a multi-core optimized recurrent neural network

Embodiments of end-to-end deep learning systems and methods are disclosed to recognize speech of vastly different languages, such as English or Mandarin Chinese. In embodiments, the entire pipelines of hand-engineered components are replaced with neural networks, and the end-to-end learning allows handling a diverse variety of speech including noisy environments, accents, and different languages. Using a trained embodiment and an embodiment of a batch dispatch technique with GPUs in a data center, an end-to-end deep learning system can be inexpensively deployed in an online setting, delivering low latency when serving users at scale.

End-to-end speech recognition

Deployed end-to-end speech recognition

Growing community of inventors

San Francisco, CA, United States of America

Dario Amodei