Apple Inc.

Ganesh G Yadav

Sorin Constantin Cismas

Jim Chen Chou

Hao Pan

Krishnakanth Rapaka

Jiandong Shen

Munehiro Nakazato

The present disclosure relates to systems and methods of multi-pipe scheduling for image decoding. For example, a bitstream may include compressed slices that are scheduled to each of the multiple decoding pipelines present in the image processing circuitry of an electronic device. The bitstream may include image data that was encoded using variable-length coding. This results in some bits of the image data containing denser and/or sparser syntax elements and result in variable processing times. The scheduling circuitry may be able to monitor each of the multiple decoder pipelines rate of bit consumption and the number of bits initially scheduled to each of the multiple pipelines and the number of bits consumed by each of the multiple pipelines over time so that incoming compressed slices may be scheduled to the pipe that will be able to process the compressed slice the fastest.

Systems and methods for multi-pipe scheduling during image decoding

This disclosure is directed to systems and methods of rate control in multiple pass video encoding. The video encoder may complete multiple encoding passes for slices of an image. Rate control algorithms may be implemented that scale the quantization step size and quantization matrix values depending on the determined size of the image slices. This may enable the size of slices to be adjusted based on size parameters for the image data.

Systems and methods of rate control for multiple pass video encoding

This disclosure is directed to systems and methods of data partitioning in image encoding. A video encoder may receive a single stream of video data that includes multiple layers. The video encoder may encode the one or more layers utilizing multiple counters for each component within the layers. The multiple counters may correspond to the header bits, luma bits, and chroma bits within each slice layer. The encoded layers may then be assembled into a single slice before it is sent to a decoder for decoding of image frame.

Systems and methods for data partitioning in video encoding

This disclosure is directed to systems and methods of streaming extensions for video encoding. The streaming extensions may enable the bitstream syntax for layered video data to be modified to reduce overhead for encoding. The bitstream syntax may be modified to enable variable length luma and chroma components, and enable the alignment between the layers and slice to be bit aligned to enable increased granularity in image encoding, and to minimize overhead between different elements within the layers.

Systems and methods for streaming extensions for video encoding

In one implementation, a method of encoding an image is performed at a device including one or more processors and non-transitory memory. The method includes determining a category of a spatial portion of an image based on a relation between a plurality of thresholds associated with a plurality of quantization scaling parameters and a bit rate of the spatial portion of the image at the plurality of quantization scaling parameters. The method includes quantizing the spatial portion of the image based on the categorization.

Video encoder with quantization control

In one implementation, a method of encoding an image is performed at a device including one or more processors and non-transitory memory. The method includes receiving a first image comprising a plurality of pixels having a respective plurality of pixel locations and a respective plurality of pixel values. The method includes applying a frequency transform to a first spatial portion of the first image to generate a plurality of first frequency coefficients respectively associated with a plurality of spatial frequencies and applying the frequency transform to a second spatial portion of the first image to generate a plurality of second frequency coefficients respectively associated with the plurality of spatial frequencies. The method includes obtaining a first quantization matrix including, for each of the plurality of spatial frequencies, a respective first quantization parameter, wherein a first quantization parameter associated with a first spatial frequency of the plurality of spatial frequencies has a first ratio with a first quantization parameter associated with a second spatial frequency of the plurality of spatial frequencies, and obtaining a second quantization matrix including, for each of the plurality of spatial frequencies, a respective second quantization parameter, wherein a second quantization parameter associated with the first spatial frequency has a second ratio, different than the first ratio, with a second quantization parameter associated with the second spatial frequency. The method includes quantizing the plurality of first frequency coefficients according to the first quantization matrix to generate a plurality of first quantized frequency coefficients and quantizing the plurality of second frequency coefficients according to the second quantization matrix to generate a plurality of second quantized frequency coefficients. The method includes transmitting the plurality of first quantized frequency coefficients and the plurality of second quantized frequency coefficients.

Growing community of inventors

Fremont, CA, United States of America

Ganesh G Yadav