Future Wei Technologies, Inc.

Kwai Inc.

Santa Clara University

Huawei Technologies Co., Limited

Tsinghua University

Beijing Transtreams Technology Co. Ltd.

Realtek Semiconductor Inc.

Lingzhi Liu

Nam Ling

Yongxiong Ren

Jianhua Zheng

Guichun Li

Yang Liu

Changcai Lai

Chen-Xiong Zhang

Yun He

Xiaoran Cao

Ying Liu

Yongbing Lin

Heng Liu

Qiang Li

Ming Kai Hsu

Xiaorui Wang

Li Song

Bingbing Li

Yifei Pei

Cheng-Xiong Zhang

Xiaozhen Zheng

Li Liu

Jing Wang

Yuan Yuan

Weimin Zeng

Zhen Peng

Xin Chen

Yunfei Wang

Hai Chen

Chi-Wang Chai

Xiang Li

Wujun Chen

Wei Pu

Philipp Zhang

Yuanyuan Zhao

Qingxi He

Kaituo Xu

Zhendong Wang

Jishen Yang

Hanxian Huang

Pengli Du

Xing Wen

Chenxiong Zhang

Methods and apparatuses for automatic speech recognition are provided. The method includes: generating a weight matrix for a layer of a plurality of layers in a neural network; dividing the weight matrix into a plurality of blocks, each block including a plurality of weights; selecting a set of blocks from the plurality of blocks for block-wise pruning by minimizing a cost function subject to a pre-determined block-wise constraint; and generating a block-wise pruned weight matrix by setting one or more weights in the set of blocks to zero. The weight matrix includes a set of weights associated with the layer, the plurality of layers includes a first layer receiving a first input associated with one or more audio feature sequences, and the plurality of layers are executed on one or more processors.

Methods and devices for structured pruning for automatic speech recognition

Methods and apparatuses are provided for serving multiple models with a single engine. The method includes: providing a single engine for a plurality of models on a server, where the server includes at least one graphics processing unit (GPU) and memory coupled with the at least one GPU; loading the plurality of models onto the memory of the server at once; and serving, by the single engine, the plurality of models, where the single engine accommodates all structures and weights of the plurality of models with a shared input and output of the memory.

Methods and devices for multiple model single deployment engine implementation

A class-specific neural network for video compressed sensing and methods for training and testing the class-specific neural network are provided. The class-specific neural network includes a Gaussian-mixture model (GMM) and a plurality of encoders, where the GMM classifies video frame blocks with a plurality of clusters and assigns the video frame blocks to the plurality of clusters. Further, the plurality of encoders receive the video frame blocks and generate a plurality of compressed-sensed frame block vectors, where the plurality of encoders correspond to the plurality of clusters.

Class-specific neural network for video compressed sensing

A neural network system, a method and an apparatus for image compression are provided. The neural network may include a generator including an encoder, an entropy estimator, and a decoder, where the encoder receives an input image and generates an encoder output, a plurality of quantized feature entries are obtained based on the encoder output outputted at a last encoder block, the entropy estimator receives the plurality of quantized feature entries and calculates an entropy loss based on the plurality of quantized feature entries, and the decoder receives the plurality of quantized feature entries and generates a reconstructed image. Furthermore, the neural network may include a discriminator that determines whether the reconstructed image different from the input image based on a discriminator loss. Moreover, the generator may determine whether content of the reconstructed image matches content of the input image based on a generator loss including the entropy loss.

End-to-end deep generative network for low bitrate image coding

A method and an apparatus for video processing are provided. The method includes that a decoding terminal receives a plurality of coded video frames coded using one or more generative adversarial networks (GANs), receives network parameters related to the one or more GANs, and decodes the plurality of coded video frames using GANs based on the network parameters. Further, the one or more GANs respectively implement one or more video coding functions including reference-frame coding, motion-compensated frame prediction, and residue-frame coding.

Generative adversarial network for video compression

A method and an apparatus for length-aware local tiling in a sparse attention module in a transformer in heterogeneous devices are provided. The method includes that a heterogeneous device including one or more GPUs: divides a transformed sparsity mask into a plurality of first tiles and obtaining one or more effective first tiles from the plurality of first tiles, where each effective first tile includes at least one non-zero element; loads the one or more effective first tiles into a shared memory in the one or more GPUs and loads a plurality of elements in a first matrix corresponding to the one or more effective first tiles into the shared memory; and performs multiplication by a first sampled dense-dense matrix multiplication (SDDMM) kernel in the sparse attention module in the transformer by fetching the one or more effective first tiles and the plurality of elements from the shared memory.

Method and apparatus for length-aware local tiling in a sparse attention module in a transformer

A method and an apparatus for automatic speech recognition are provided. The method includes: generating a weight matrix for a layer of a plurality of layers in a neural network; dividing the weight matrix into a plurality of blocks, each block including a plurality of weights; selecting a pre-determined percentage of weights from at least one block for block-wise pruning; and generating a block-wise pruned weight matrix by setting the pre-determined percentage of weights selected from the at least one block to zero. The weight matrix includes a set of weights associated with the layer, the plurality of layers includes a first layer receiving a first input associated with one or more audio feature sequences, and the plurality of layers are executed on one or more processors. The method efficiently accelerates model inference using irregular pruning.

Methods and devices for irregular pruning for automatic speech recognition

A method, apparatus, and a non-transitory computer-readable storage medium for generating heterogenous platform code. The method may obtain a neural network model. The neural network model may be programed to run on at least one platform. The method may also obtain an initial intermediate representation (IR) code by encoding the neural network model, and obtain a target IR code by adding decorations to the initial IR code based on a target platform. The method may also output an executable code optimized to run on the target platform by decoding the target IR code.

Multi-level intermediate representation decoder for heterogeneous platforms

Systems and methods are provided for automatic speech recognition. In the method, the system obtains a padded sequence by processing a plurality of acoustic signals. The system compresses the padded sequence by reducing the size of the padded sequence to obtain a compressed sequence. The system inputs the compressed sequence into a pre-trained encoder neural network to obtain an encoded sequence and then decompresses the encoded sequence by recovering the encoded sequence to an original sequential ordering. The system inputs the encoded sequence to a decoding module to obtain recognition texts.

Systems and methods for accelerating automatic speech recognition based on compression and decompression

An encoding method with multiple image block division manners is disclosed, including: determining a division manner and a division direction of an image block; dividing the image block to obtain image subblocks sequentially arranged horizontally or vertically; determining whether the image subblocks need subdivision, and if subdivision is not needed, predicting the encoding object in the frame according to the image subblocks, to obtain residual data; performing transformation, quantization, and entropy encoding for the residual data so as to obtain coded residual data; and writing the division manner of the image block, the division direction of the image block, an identifier indicating whether the image subblocks need subdivision, and the coded residual data into a bitstream. By applying the encoding method, better prediction accuracy can be achieved when the image block presents a small change of pixel value in the horizontal or vertical direction.

Encoding and decoding methods and apparatuses with multiple image block division manners

An automatic speech recognition system and a method thereof are provided. The system includes an encoder and a decoder. The encoder comprises a plurality of encoder layers. At least one encoder layer includes a plurality of encoder sublayers fused into one or more encoder kernels. The system further comprises a first pair of ping-pong buffers communicating with the one or more encoder kernels. The decoder comprises a plurality of decoder layers. At least one decoder layer includes a plurality of decoder sublayers fused into one or more decoder kernels. The decoder receives a decoder output related to the encoder output and generates a decoder output. The encoder sends the decoder output to a beam search kernel.

Systems and methods for automatic speech recognition based on graphics processing units

The present disclosure relates to an automatic speech recognition system and a method thereof. The system includes a conformer encoder and a pair of ping-pong buffers. The encoder includes a plurality of encoder layers sequentially executed by one or more graphic processing units. At least one encoder layer includes a first feed forward module, a multi-head self-attention module, a convolution module, and a second feed forward module. The convolution module and the multi-head self-attention module are sandwiched between the first feedforward module and the second feed forward module. The four modules respectively include a plurality of encoder sublayers fused into one or more encoder kernels. The one or more encoder kernels respectively read from one of the pair of ping-pong buffers and write into the other of the pair of ping-pong buffers.

An apparatus including a memory operably coupled to a processor. The processor is configured to determine whether to use an intra smoothing filter for a rectangular prediction unit (PU), wherein a width of the rectangular PU is different from a height of the rectangular PU.

Mode dependent intra smoothing filter table mapping methods for non-square prediction units

A transmitter for handling multiple formats of a video sequence, comprises a preprocessing module, for receiving a first format of a video sequence, to generate metadata of a second format of the video sequence according to the first format of the video sequence and the second format of the video sequence; and an encoder, couple to the preprocessing module, for transmitting the first format of the video sequence and the metadata in a bit stream to a receiver.

Device and method of handling multiple formats of a video sequence

Growing community of inventors

San Jose, CA, United States of America

Lingzhi Liu