Asapp, Inc.

Kyu Jeong Han

Felix Wu

Tao Ma

Jing Pan

Kwangyoun Kim

Prashant Sridhar

Kilian Quirin Weinberger

Daniel Povey

Madison Chandler Riley

Yoav Artzi

Suwon Shon

Pablo Brusco

For any application that processes speech, improving the quality of the feature vectors may improve the quality of the speech application. The quality of feature vectors may be improved by modifying a neural network architecture for computing feature vectors to allocate computational resources where they are more effective for learning and computing the feature vectors. Contextual feature vectors may be computed from feature vectors by using a parameterized downsampling operation that decreases a vector sequence rate, processing the downsampled vectors with a neural network, and using a parameterized upsampling operation that increases a vector sequence rate. For example, parameterized downsampling may decrease a vector sequence rate by a factor of two, a neural may require fewer computational resources since it operates with a lower vector sequence rate, and parameterized upsampling may then increase the vector sequence rate by a factor of two.

Contextual feature vectors for processing speech

The amount of future context used in a speech processing application allows for tradeoffs between performance and the delay in providing results to users. Existing speech processing applications may be trained with a specified future context size and perform poorly when used in production with a different future context size. A speech processing application trained using a stochastic future context allows a trained neural network to be used in production with different amounts of future context. During an update step in training, a future-context size may be sampled from a probability distribution, used to mask a neural network, and compute an output of the masked neural network. The output may then be used to compute a loss value and update parameters of the neural network. The trained neural network may then be used in production with different amounts of future context to provide greater flexibility for production speech processing applications.

Stochastic future context for speech processing

Audio signals of speech may be processed using an acoustic model. An acoustic model may be implemented with multiple streams of processing where different streams perform processing using different dilation rates. For example, a first stream may process features of the audio signal with one or more convolutional neural network layers having a first dilation rate, and a second stream may process features of the audio signal with one or more convolutional neural network layers having a second dilation rate. Each stream may compute a stream vector, and the stream vectors may be combined to a vector of speech unit scores, where the vector of speech unit scores provides information about the acoustic content of the audio signal. The vector of speech unit scores may be used for any appropriate application of speech, such as automatic speech recognition.

Multistream acoustic models with dilations

The present disclosure describes a system, method, and computer program for predicting sentiment labels for audio speech utterances using an audio speech sentiment classifier pretrained with pseudo sentiment labels. A speech sentiment classifier for audio speech ('a speech sentiment classifier') is pretrained in an unsupervised manner by leveraging a pseudo labeler previously trained to predict sentiments for text. Specifically, a text-trained pseudo labeler is used to autogenerate pseudo sentiment labels for the audio speech utterances using transcriptions of the utterances, and the speech sentiment classifier is trained to predict the pseudo sentiment labels given corresponding embeddings of the audio speech utterances. The speech sentiment classifier is then subsequently fine tuned using a sentiment-annotated dataset of audio speech utterances, which may be significantly smaller than the unannotated dataset used in the unsupervised pretraining phase.

Speech sentiment analysis using a speech sentiment classifier pretrained with pseudo sentiment labels

The present disclosure relates to a system, method, and computer program for creating a complete transcription of an audio recording from separately transcribed redacted and unredacted words. The system receives an original audio recording and redacts a plurality of words from the original audio recording to obtain a modified audio recording. The modified audio recording is outputted to a first transcription service. Audio clips of the redacted words from the original audio recording are extracted using word-level timestamps for the redacted words. The extracted audio clips are outputted to a second transcription service. The system receives a transcription of the modified audio recording from the first transcription service and transcriptions of the extracted audio clips from the second transcription service. The transcriptions of the extracted audio clips are combined with the transcription of the modified audio recording to obtain a complete transcription of the original audio recording using word-level timestamps.

System, method, and computer program for creating a complete transcription of an audio recording from separately transcribed redacted and unredacted words

Growing community of inventors

Pleasanton, CA, United States of America

Kyu Jeong Han