Hong Kong Applied Science and Technology Research Institute Company, Limited

Xuejiao Liu

Yu Liu

Luhong Liang

Tao Li

King Hung Chiu

Peng Luo

Ka Lung Tim Wong

Kwok Wai Hung

Tim Ka Lung Wong

Xiaoqing Shan

A dynamic-tile neural network accelerator allows for the number and size of computational tiles to be re-configured. Each sub-array of computational cells has edge cells on the left-most column that have an added vector mux that feeds the cell output back to an adder-comparator to allow Rectified Linear Unit (ReLU) and pooling operations that combine outputs shifted in from other cells. The edge cells drive external output registers and receive external weights. The weights and outputs are shifted in opposite directions horizontally between cells while control and input data are shifted in a same direction vertically between cells. A column of row data selectors is inserted between sub-arrays to bypass weights and output data around sub-arrays, while a row of column data selectors are inserted between sub-arrays to bypass control and input data. Larger tiles are configured by passing data directly through these selectors without bypassing.

Dynamic tile parallel neural network accelerator

A neural network accelerator reads encoded weights from memory. All 1 bits in a weight except the first three are discarded. The first three leading 1 bits in the weight are encoded as three bit-shift values to form the encoded weight. The three bit-shift values are applied to a bit shifter to shift a node input to obtain three shifted inputs that are accumulated to generate the node output. Node complexity is reduced since only 3 shifts are performed rather than up to 15 shifts for a 16-bit weight. The bit shifter and accumulator for a node can be implemented by Look-Up Tables (LUTs) without requiring a Multiply-Accumulate (MAC) cell in a Field-Programmable Gate Array (FPGA). Quantization bias is reduced using a histogram analyzer that determines a weighted average for each interval between quantized weights. The third bit-shift value is incremented for weights in the interval above the weighted average.

Residual quantization of bit-shift weights in an artificial neural network

A hybrid-resolution panoramic VR generator places High-Resolution (HR) patches from a ring of HR cameras onto a 360-degree Low-Resolution (LR) image from a LR camera pointing upward from the ring into a panoramic mirror that captures the combined field of view of all the multiple HR cameras, but at a lower resolution. Ghosting artifacts caused by parallax errors between adjacent HR cameras are eliminated because object placement is determined by the 360-degree LR image. Each HR image is homographicly projected into 3 projections by grouping objects of different depths to obtain homographic matrixes. The 360-degree LR image is upscaled to HR and a query patch is searched in search windows in the three projections for up to two adjacent HR images. Best-matching patches are weighted by similarity with the query patch and blended to generate a reconstructed patch placed at the query patch location in a reconstructed HR panorama image.

High resolution (HR) panorama generation without ghosting artifacts using multiple HR images mapped to a low resolution 360-degree image

A frame-rate converter reduces halo artifacts along edges of moving objects. Halo artifacts occur on interpolated frames where a moving object covers and uncovers pixels along its edges. Motion estimation among three original frames produces hybrid direction motion vectors that are bi-directional for background and objects, but are unidirectional for covered and uncovered regions, since motion vectors with large matching errors are deleted. Covered regions in the interpolated frame are detected as intersecting only a forward but no backward hybrid motion vector. Bi-directional motion estimation from the hybrid motion vectors of two original frames produces refined motion vectors for the interpolated frame. Refined motion vectors in the covered regions are deleted and replaced with hybrid motion vectors from the original frames. Hybrid motion vectors from the original frames are assigned to the critical covered regions rather than using interpolated vectors in the covered regions, reducing halo artifacts.

Growing community of inventors

Shenzhen, China

Xuejiao Liu