Microsoft Technology Licensing, LLC

Xie Chen

Jinyu Li

Shujie Liu

Yu Wu

Chengyi Wang

Dong Yu

Rui Zhao

Yifan Gong

Gang Li

Sarangarajan Parthasarathy

Yu Shi

Nanshan Zeng

Frank Torsten Bernd Seide

Zhong Meng

Dimitrios Basile Dimitriadis

Adam Clyde Eversole

Yashesh Gaur

Liang Lu

Naoyuki Kanda

Liyang Lu

Xie Sun

The computing system trains an end-to-end (E2E) automatic speech recognition (ASR) model, using a transformer-transducer-based deep neural network that comprises a transformer encoder network and a transducer predictor network. The E2E ASR model is trained to have one or more adjustable hyperparameters that are configured to dynamically adjust an efficiency or a performance of the E2E ASR model when the E2E ASR model is deployed onto a device or executed by the device, by identifying one or more conditions of the device associated with computational power of the device and setting at least one of the one or more adjustable hyperparameters based on one or more conditions of the device.

Efficiency adjustable speech recognition system

A computing system is configured to generate a transformer-transducer-based deep neural network. The transformer-transducer-based deep neural network comprises a transformer encoder network and a transducer predictor network. The transformer encoder network has a plurality of layers, each of which includes a multi-head attention network sublayer and a feed-forward network sublayer. The computing system trains an end-to-end (E2E) automatic speech recognition (ASR) model, using the transformer-transducer-based deep neural network. The E2E ASR model has one or more adjustable hyperparameters that are configured to dynamically adjust an efficiency or a performance of E2E ASR model when the E2E ASR model is deployed onto a device or executed by the device.

A computer device is provided that includes one or more processors configured to receive an end-to-end (E2E) model that has been trained for automatic speech recognition with training data from a source-domain, and receive an external language model that has been trained with training data from a target-domain. The one or more processors are configured to perform an inference of the probability of an output token sequence given a sequence of input speech features. Performing the inference includes computing an E2E model score, computing an external language model score, and computing an estimated internal language model score for the E2E model. The estimated internal language model score is computed by removing a contribution of an intrinsic acoustic model. The processor is further configured to compute an integrated score based at least on E2E model score, the external language model score, and the estimated internal language model score.

Internal language model for E2E models

According to some embodiments, a multi-layer speech recognition transcript post processing system may include a data-driven, statistical layer associated with a trained automatic speech recognition model that selects an initial transcript. A rule-based layer may receive the initial transcript from the data-driven, statistical layer and execute at least one pre-determined rule to generate a first modified transcript. A machine learning approach layer may receive the first modified transcript from the rule-based layer and perform a neural model inference to create a second modified transcript. A human editor layer may receive the second modified transcript from the machine learning approach layer along with an adjustment from at least one human editor. The adjustment may create, in some embodiments, a final transcript that may be used to fine-tune the data-driven, statistical layer.

Data-driven and rule-based speech recognition output enhancement

The use of a pipelined algorithm that performs parallelized computations to train deep neural networks (DNNs) for performing data analysis may reduce training time. The DNNs may be one of context-independent DNNs or context-dependent DNNs. The training may include partitioning training data into sample batches of a specific batch size. The partitioning may be performed based on rates of data transfers between processors that execute the pipelined algorithm, considerations of accuracy and convergence, and the execution speed of each processor. Other techniques for training may include grouping layers of the DNNs for processing on a single processor, distributing a layer of the DNNs to multiple processors for processing, or modifying an execution order of steps in the pipelined algorithm.

Deep neural networks training for speech and pattern recognition

Growing community of inventors

Bellevue, WA, United States of America

Xie Chen