Ati Technologies Ulc

Advanced Micro Devices Corporation

International Business Machines Corporation

Lei Zhang

Gabor Sines

Khaled Mammou

Edward A Harold

Ihab Amer

David I Glen

Zhen Biao Chen

Ji Zhou

Sateesh Lagudu

Layla A Mah

Bruce Montag

Haibo Liu

Allen H Rush

Rajabali M Koduri

Min Yu

Yang Liu

Ihab M A Amer

Jiao Wang

Fabio Gulino

Benedict Chien

Zhiqi Hao

Michael L Schmit

Daniel Wong

John Wright Stephenson

Ying Luo

Andy Sung

Baochun Li

Ho Hin Lau

Jinbo Qiu

Ehsan Mirhadi

Trevett B Chusing

Zhong Hai Cai

Jian Zhu

Samuel D Naffziger

Boris Ivanovic

Syed Athar Hussain

Mehdi Saeedi

Guennadi Riguer

Kostantinos Danny Christidis

Ahmed M Abdelkhalek

Shu-Hsien Wu

Mohamed K Cherif

Ping Chen

Arash Hariri

Haibin Li

Stephen Ho

Razvan Florin Dan-Dobre

Purushotham Niranjan Dasiga

Dennis Lew

Lawrence A Bair

Stephen A J Bagshaw

Naveed A Gazi

Steven Lok-Man Doo

Jonathon Walter Riley

John-Paul A Compagnone

Li Baochun

Gerald S C Chan

Syed Y Ali

Zhengang Liang

Kai Sun

Kim A Meinerth

Yi Wang

Ihab MA Amer

John-Paul Compagnone

Gerald Sc Chan

Zhen Chen

A device and method for processing Virtual Reality (VR) data is disclosed. The method comprises transmitting feedback information from the device to a server, wherein the feedback information is captured in the device, receiving data from the server to be presented on the device based on the feedback information, wherein the data includes video data and audio data where the video data is a frame of video data in a sequence of frames and the audio data is the corresponding audio data of the frame, decoding the video data and corresponding audio data of the frame, and controlling the presentation of the video data and corresponding audio data on the device such that the video data is synchronized with the corresponding audio data.

Low latency wireless virtual reality systems and methods

Systems, apparatuses, and methods for adaptively mapping a machine learning model to a multi-core inference accelerator engine are disclosed. A computing system includes a multi-core inference accelerator engine with multiple inference cores coupled to a memory subsystem. The system also includes a control unit which determines how to adaptively map a machine learning model to the multi-core inference accelerator engine. In one implementation, the control unit selects a mapping scheme which minimizes the memory bandwidth utilization of the multi-core inference accelerator engine. In one implementation, this mapping scheme involves having one inference core of the multi-core inference accelerator engine fetch given data and broadcast the given data to other inference cores of the inference accelerator engine. Each inference core fetches second data unique to the respective inference core. The inference cores then perform computations on the first and second data in order to implement the machine learning model.

Machine learning inference engine scalability

A system configured to perform scalable video encoding is provided. The system includes a memory; and a processing unit, wherein the processing unit is configured to: receive inter-layer data and a current picture, wherein the current picture has a base layer; upsample the inter-layer data to generate residual data and reconstruction data, wherein the inter-layer data includes a base mode flag; and encode the current picture to an enhanced layer using the upsampled inter-layer data based on a block type of the base layer and the base mode flag.

Bandwidth saving architecture for scalable video coding

Systems, apparatuses, and methods for implementing a low latency long short-term memory (LSTM) machine learning engine using sequence interleaving techniques are disclosed. A computing system includes at least a host processing unit, a machine learning engine, and a memory. The host processing unit detects a plurality of sequences which will be processed by the machine learning engine. The host processing unit interleaves the sequences into data blocks and stores the data blocks in the memory. When the machine learning engine receives a given data block, the machine learning engine performs, in parallel, a plurality of matrix multiplication operations on the plurality of sequences in the given data block and a plurality of coefficients. Then, the outputs of the matrix multiplication operations are coupled to one or more LSTM layers.

Low latency long short-term memory inference with sequence interleaving

Virtual Reality (VR) processing devices and methods are provided for transmitting user feedback information comprising at least one of user position information and user orientation information, receiving encoded audio-video (A/V) data, which is generated based on the transmitted user feedback information, separating the A/V data into video data and audio data corresponding to a portion of a next frame of a sequence of frames of the video data to be displayed, decoding the portion of a next frame of the video data and the corresponding audio data, providing the audio data for aural presentation and controlling the portion of the next frame of the video data to be displayed in synchronization with the corresponding audio data.

A device includes an encoder, decoder, codec or combination thereof and inline hardware conversion units that are operative to convert stored image data into one of: an HDR/WCG format and an SDR/SCG format during the conversion process. Each of the inline hardware conversion units is operative to perform the conversion process independent of another read operation with the memory that stores the image data to be converted. In one example, an encoding unit is operative to perform a write operation with a memory to store the converted image data after completing the conversion process. In another example, a decoding unit is operative to perform a read operation with the memory to retrieve the image data from the memory before initiating the conversion process. In another example, an encoder/decoder unit is operative to perform at least one of: the read operation and the write operation.

Method and apparatus for an HDR hardware processor inline to hardware encoder and decoder

Systems, apparatuses, and methods for generating a model for determining a quantization strength to use when encoding video frames are disclosed. A pre-encoder performs multiple encoding passes using different quantization strengths on a portion or the entirety of one or more pre-processed video frames. The pre-encoder captures the bit-size of the encoded output for each of the multiple encoding passes. Then, based on the multiple encoding passes, the pre-encoder generates a model for mapping bit-size to quantization strength for encoding video frames or portion(s) thereof. When the encoder begins the final encoding pass for one or more given video frames or any portion(s) thereof, the encoder uses the model to map a preferred bit-size to a given quantization strength. The encoder uses the given quantization strength when encoding the given video frame(s) or frame portion(s) to meet a specified bit-rate for the encoded bitstream.

Content adaptive quantization strength and bitrate modeling

Systems, apparatuses, and methods for implementing a broadcast read response protocol are disclosed. A computing system includes a plurality of processing engines coupled to a memory subsystem. A first processing engine executes a read and broadcast response command, wherein the read and broadcast response command targets first data at a first address in the memory subsystem. One or more other processing engines execute a wait command to wait to receive the first data requested by the first processing engine. After receiving the first data from the memory subsystem, the plurality of processing engines process the first data as part of completing a first operation. In one implementation, the first operation is implementing a given layer of a machine learning model. In one implementation, the given layer is a convolutional layer of a neural network.

Broadcast command and response

An apparatus and methods for controlling power consumption in video encoding obtain, before motion estimation is performed on an image frame to be encoded, information regarding an amount of the image frame to be encoded that is static with respect to a previously encoded image frame. The apparatus and methods adjust power consumption of the video encoder based on the obtained information regarding the amount of the image frame to be encoded that is static.

Controlling power consumption in video encoding based on information regarding static amount of an image frame

The present disclosure is directed a system and method for exploiting camera and depth information associated with rendered video frames, such as those rendered by a server operating as part of a cloud gaming service, to more efficiently encode the rendered video frames for transmission over a network. The method and system of the present disclosure can be used in a server operating in a cloud gaming service to improve, for example, the amount of latency, downstream bandwidth, and/or computational processing power associated with playing a video game over its service. The method and system of the present disclosure can be further used in other applications where camera and depth information of a rendered or captured video frame is available.

Exploiting camera depth information for video encoding

Systems, apparatuses, and methods for reducing latency when consuming an encoded video bitstream in real-time are disclosed. A video encoder encodes a video bitstream and writes chunks of the encoded bitstream to a bitstream buffer. Prior to the encoder completing the encoding of an entire frame, or an entire slice of a frame, a consumer module consumes encoded chunks of the bitstream. In one implementation, to enable pipelining of the consumption with the encoding, the encoder updates a buffer write pointer with an indication of the amount of data that has been written to the bitstream buffer. The consumer module retrieves encoded data from the bitstream buffer up to the location indicated by the buffer write pointer. In this way, the consumer module is able to access and consume encoded video data prior to the encoder finishing encoding an entire frame or an entire slice of the frame.

Low-latency consumption of an encoded video bitstream

Systems, apparatuses, and methods for implementing memory bandwidth reduction techniques for low power convolutional neural network inference applications are disclosed. A system includes at least a processing unit and an external memory coupled to the processing unit. The system detects a request to perform a convolution operation on input data from a plurality of channels. Responsive to detecting the request, the system partitions the input data from the plurality of channels into 3D blocks so as to minimize the external memory bandwidth utilization for the convolution operation being performed. Next, the system loads a selected 3D block from external memory into internal memory and then generates convolution output data for the selected 3D block for one or more features. Then, for each feature, the system adds convolution output data together across channels prior to writing the convolution output data to the external memory.

Memory bandwidth reduction techniques for low power convolutional neural network inference applications

A system and method for scalable video coding that includes base layer having lower resolution encoding, enhanced layer having higher resolution encoding and the data transferring between two layers. The system and method provides several methods to reduce bandwidth of inter-layer transfers while at the same time reducing memory requirements. Due to less memory access, the system clock frequency can be lowered so that system power consumption is lowered as well. The system avoids having prediction data from base layer to enhanced layer to be up-sampled for matching resolution in the enhanced layer as transferring up-sampled data can impose a big burden on memory bandwidth.

Systems, apparatuses, and methods for calculating a quantization parameter (QP) for encoding video frames to meet a given bit budget are disclosed. Control logic coupled to an encoder calculates a complexity indicator that represents a level of difficulty in encoding a previous video frame. The complexity indicator is based at least in part on a first parameter associated with the previous video frame and corresponds to one or more of a variance, an intra-prediction factor, and an inter-to-intra ratio. The complexity indicator is then used by the control logic to calculate a preferred QP to use to encode the current video frame to meet a given bit budget. By using the preferred QP generated based on the complexity indicator, the encoder is able to make fewer QP adjustments during the frame. This helps to improve the visual quality of the resulting encoded video bitstream.

Efficient quantization parameter prediction method for low latency video coding

Low latency wireless Virtual Reality systems and methods

Growing community of inventors

Richmond Hill, Canada

Lei Zhang