Kwai Inc.

Gyrfalcon Technology Inc.

Beijing Transtreams Technology Co. Ltd.

Santa Clara University

Yongxiong Ren

Lingzhi Liu

Yang Liu

Lin Yang

Yequn Zhang

Ying Liu

Baohua Sun

Nam Ling

Heng Liu

Qiang Li

Qi Dong

Ming Kai Hsu

Xiaochun Li

Xiaorui Wang

Bin Yang

Bingbing Li

Yifei Pei

Patrick Dong

Yinbo Shi

Yuanyuan Zhao

Yi Fan

Kaituo Xu

Zhendong Wang

Jishen Yang

Hanxian Huang

Pengli Du

Xing Wen

Methods and apparatuses for automatic speech recognition are provided. The method includes: generating a weight matrix for a layer of a plurality of layers in a neural network; dividing the weight matrix into a plurality of blocks, each block including a plurality of weights; selecting a set of blocks from the plurality of blocks for block-wise pruning by minimizing a cost function subject to a pre-determined block-wise constraint; and generating a block-wise pruned weight matrix by setting one or more weights in the set of blocks to zero. The weight matrix includes a set of weights associated with the layer, the plurality of layers includes a first layer receiving a first input associated with one or more audio feature sequences, and the plurality of layers are executed on one or more processors.

Methods and devices for structured pruning for automatic speech recognition

Methods and apparatuses are provided for serving multiple models with a single engine. The method includes: providing a single engine for a plurality of models on a server, where the server includes at least one graphics processing unit (GPU) and memory coupled with the at least one GPU; loading the plurality of models onto the memory of the server at once; and serving, by the single engine, the plurality of models, where the single engine accommodates all structures and weights of the plurality of models with a shared input and output of the memory.

Methods and devices for multiple model single deployment engine implementation

A class-specific neural network for video compressed sensing and methods for training and testing the class-specific neural network are provided. The class-specific neural network includes a Gaussian-mixture model (GMM) and a plurality of encoders, where the GMM classifies video frame blocks with a plurality of clusters and assigns the video frame blocks to the plurality of clusters. Further, the plurality of encoders receive the video frame blocks and generate a plurality of compressed-sensed frame block vectors, where the plurality of encoders correspond to the plurality of clusters.

Class-specific neural network for video compressed sensing

A neural network system, a method and an apparatus for image compression are provided. The neural network may include a generator including an encoder, an entropy estimator, and a decoder, where the encoder receives an input image and generates an encoder output, a plurality of quantized feature entries are obtained based on the encoder output outputted at a last encoder block, the entropy estimator receives the plurality of quantized feature entries and calculates an entropy loss based on the plurality of quantized feature entries, and the decoder receives the plurality of quantized feature entries and generates a reconstructed image. Furthermore, the neural network may include a discriminator that determines whether the reconstructed image different from the input image based on a discriminator loss. Moreover, the generator may determine whether content of the reconstructed image matches content of the input image based on a generator loss including the entropy loss.

End-to-end deep generative network for low bitrate image coding

A method and an apparatus for video processing are provided. The method includes that a decoding terminal receives a plurality of coded video frames coded using one or more generative adversarial networks (GANs), receives network parameters related to the one or more GANs, and decodes the plurality of coded video frames using GANs based on the network parameters. Further, the one or more GANs respectively implement one or more video coding functions including reference-frame coding, motion-compensated frame prediction, and residue-frame coding.

Generative adversarial network for video compression

A method and an apparatus for length-aware local tiling in a sparse attention module in a transformer in heterogeneous devices are provided. The method includes that a heterogeneous device including one or more GPUs: divides a transformed sparsity mask into a plurality of first tiles and obtaining one or more effective first tiles from the plurality of first tiles, where each effective first tile includes at least one non-zero element; loads the one or more effective first tiles into a shared memory in the one or more GPUs and loads a plurality of elements in a first matrix corresponding to the one or more effective first tiles into the shared memory; and performs multiplication by a first sampled dense-dense matrix multiplication (SDDMM) kernel in the sparse attention module in the transformer by fetching the one or more effective first tiles and the plurality of elements from the shared memory.

Method and apparatus for length-aware local tiling in a sparse attention module in a transformer

A method and an apparatus for automatic speech recognition are provided. The method includes: generating a weight matrix for a layer of a plurality of layers in a neural network; dividing the weight matrix into a plurality of blocks, each block including a plurality of weights; selecting a pre-determined percentage of weights from at least one block for block-wise pruning; and generating a block-wise pruned weight matrix by setting the pre-determined percentage of weights selected from the at least one block to zero. The weight matrix includes a set of weights associated with the layer, the plurality of layers includes a first layer receiving a first input associated with one or more audio feature sequences, and the plurality of layers are executed on one or more processors. The method efficiently accelerates model inference using irregular pruning.

Methods and devices for irregular pruning for automatic speech recognition

A method, apparatus, and a non-transitory computer-readable storage medium for generating heterogenous platform code. The method may obtain a neural network model. The neural network model may be programed to run on at least one platform. The method may also obtain an initial intermediate representation (IR) code by encoding the neural network model, and obtain a target IR code by adding decorations to the initial IR code based on a target platform. The method may also output an executable code optimized to run on the target platform by decoding the target IR code.

Multi-level intermediate representation decoder for heterogeneous platforms

Systems and methods are provided for automatic speech recognition. In the method, the system obtains a padded sequence by processing a plurality of acoustic signals. The system compresses the padded sequence by reducing the size of the padded sequence to obtain a compressed sequence. The system inputs the compressed sequence into a pre-trained encoder neural network to obtain an encoded sequence and then decompresses the encoded sequence by recovering the encoded sequence to an original sequential ordering. The system inputs the encoded sequence to a decoding module to obtain recognition texts.

Systems and methods for accelerating automatic speech recognition based on compression and decompression

An automatic speech recognition system and a method thereof are provided. The system includes an encoder and a decoder. The encoder comprises a plurality of encoder layers. At least one encoder layer includes a plurality of encoder sublayers fused into one or more encoder kernels. The system further comprises a first pair of ping-pong buffers communicating with the one or more encoder kernels. The decoder comprises a plurality of decoder layers. At least one decoder layer includes a plurality of decoder sublayers fused into one or more decoder kernels. The decoder receives a decoder output related to the encoder output and generates a decoder output. The encoder sends the decoder output to a beam search kernel.

Systems and methods for automatic speech recognition based on graphics processing units

The present disclosure relates to an automatic speech recognition system and a method thereof. The system includes a conformer encoder and a pair of ping-pong buffers. The encoder includes a plurality of encoder layers sequentially executed by one or more graphic processing units. At least one encoder layer includes a first feed forward module, a multi-head self-attention module, a convolution module, and a second feed forward module. The convolution module and the multi-head self-attention module are sandwiched between the first feedforward module and the second feed forward module. The four modules respectively include a plurality of encoder sublayers fused into one or more encoder kernels. The one or more encoder kernels respectively read from one of the pair of ping-pong buffers and write into the other of the pair of ping-pong buffers.

A system for training an artificial intelligence (AI) model for an AI chip to implement an AI task may include an AI training unit to train weights of an AI model in floating point, a convolution quantization unit for quantizing the trained weights to a number of quantization levels, and an activation quantization unit for updating the weights of the AI model so that output of the AI model based at least on the updated weights are within a range of activation layers of the AI chip. The updated weights may be stored in fixed point and uploadable to the AI chip. The various units may be configured to account for the hardware constraints in the AI chip to minimize performance degradation when the trained weights are uploaded to the AI chip and expedite training convergence. Forward propagation and backward propagation may be combined in training the AI model.

Using quantization in training an artificial intelligence model in a semiconductor solution

A system may include multiple client devices and a processing device communicatively coupled to the client devices. Each client device includes an artificial intelligence (AI) chip and is configured to generate an AI model. The processing device may be configured to (i) receive a respective AI model and an associated performance value of the respective AI model from each of the plurality of client devices; (ii) determine an optimal AI model based on the performance values associated with the respective AI models from the plurality of client devices; and (iii) determine a global AI model based on the optimal AI model. The system may load the global AI model into an AI chip of a client device to cause the client device to perform an AI task based on the global AI model in the AI chip. The AI model may include a convolutional neural network.

Systems and methods for determining an artificial intelligence model in a communication system

In some embodiments, a system includes an artificial intelligence (AI) chip and a processor coupled to the AI chip and configured to receive an input image, crop the input image into a plurality of cropped images, and execute the AI chip to produce a plurality of feature maps based on at least a subset of the plurality of cropped images. The system may further merge at least a subset of the plurality of feature maps to form a merged feature map, and produce an output image based on the merged feature map. The cropping and merging operations may be performed according to a same pattern. The system may also include a training network configured to train weights of the CNN model in the AI chip in a gradient descent network. Cropping and merging may be performed over the training sample images in the training work in a similar manner.

Combining feature maps in an artificial intelligence semiconductor solution

A device for obtaining a local optimal AI model may include an artificial intelligence (AI) chip and a processing device configured to receive a first initial AI model from the host device. The device may load the initial AI model into the AI chip to determine a performance value of the AI model based on a dataset, and determine a probability that a current AI model should be replaced by the initial AI model. The device may determine, based on the probability, whether to replace the current AI model with the initial AI model. If it is determined that the current AI model be replaced, the device may replace the current AI model with the initial AI model. The device may repeat the above processes and obtain a final current AI model. The device may transmit the final current AI model to the host device.

Growing community of inventors

San Jose, CA, United States of America

Yongxiong Ren