Apple Inc.

Weichun Ku

Guy Cote

Jim Chen Chou

Suk Hwan Lim

D Amnon Silverstein

Joseph P Bratt

Mark Hayter

Zongjian Chen

Graham D Finlayson

Dominic Go

Steven David Hordley

Timothy John Millet

Sheng Lin

Muge Wang

Manching Ko

Hsiao-Chen Chang

Syed Muhammad Ali Rizvi

Yixiang Tao

Mark D Hayter

Zongijan Chen

Embodiments of the present disclosure relate to generating motion vectors. An image signal processor includes a statistics circuit and a vector correlation analysis circuit. The statistics circuit determines image statistics such as vectors representing sums of pixel values of rows or columns of blocks of an image. Additionally, the statistics circuit may mix or aggregate sums of multiple color components. The vector correlation analysis performs cross-correlation between vectors of a current image and reference vectors of a prior image to determine cross-correlation scores. The vector correlation analysis generates a motion vector by identifying shifts in horizontal and vertical directions corresponding to peak values of cross-correlation scores.

Image statistics for motion detection

System and method for improving operational efficiency of a video encoding pipeline used to encode image data. In embodiments, the video encoding pipeline includes a transcode pipeline that provides entropy encoding of binarized syntax elements. More specifically, multiple bins may be encoded in parallel, resulting in increased encoding throughput.

Parallel bypass and regular bin coding

Systems and methods for local tone mapping are provided. In one example, an electronic device includes an electronic display, an imaging device, and an image signal processor. The electronic display may display images of a first bit depth, and the imaging device may include an image sensor that obtains image data of a higher bit depth than the first bit depth. The image signal processor may process the image data, and may include local tone mapping logic that may apply a spatially varying local tone curve to a pixel of the image data to preserve local contrast when displayed on the display. The local tone mapping logic may smooth the local tone curve applied to the intensity difference between the pixel and another nearby pixel exceeds a threshold.

Systems and methods for local tone mapping

System and method for improving operational efficiency of a video encoding pipeline used to encode image data. In embodiments, the video encoding pipeline includes bit-rate statistics generation that is useful for controlling subsequent bit rates and/or determining encoding operational modes.

Rate-estimation for encoding rate control

A video encoder may include a context-adaptive binary arithmetic coding (CABAC) encode component that converts each syntax element of a representation of a block of pixels to binary code, serializes it, and codes it mathematically with its probability model, after which the resulting bit stream is output. When the probability of a bin being coded with one of two possible symbols is one-half, the bin may be coded using bypass bin coding mode rather than a more compute-intensive regular bin coding mode. The CABAC encoder may code multiple consecutive bypass bins in a series of cascaded processing units during a single processing cycle (e.g., a regular bin coding cycle). Intermediate outputs of each processing unit may be coupled to inputs of the next processing unit. A resolver unit may accept intermediate outputs of the processing units and generate final output bits for the bypass bins.

Parallel encoding of bypass binary symbols in CABAC encoder

A video encoder may include a context-adaptive binary arithmetic coding (CABAC) encode component that converts each syntax element of a representation of a block of pixels to binary code, serializes it, and codes it mathematically, after which the resulting bit stream is output. A lookup table in memory and a context cache may store probability values for supported contexts, which may be retrieved from the table or cache for use in coding syntax elements. Depending on the results of a syntax element coding, the probability value for its context may be modified (e.g., increased or decreased) in the cache and, subsequently, in the table. After coding multiple syntax elements, and based on observed access patterns for probability values, a mapping or indexing for the cache or the table may be modified to improve cache performance (e.g., to reduce cache misses or access data for related contexts using fewer accesses).

Context re-mapping in CABAC encoder

Blocks of pixels from a video frame may be encoded in a block processing pipeline using wavefront ordering, e.g. according to knight's order. Each of the encoded blocks may be written to a particular one of multiple buffers such that the blocks written to each of the buffers represent consecutive blocks of the frame in scan order. Stitching information may be written to the buffers at the end of each row. A stitcher may read the rows from the buffers in order and generate a scan order output stream for the frame. The stitcher component may read the stitching information at the end of each row and apply the stitching information to one or more blocks at the beginning of a next row to stitch the next row to the previous row. Stitching may involve modifying pixel(s) of the blocks and/or modifying metadata for the blocks.

Wavefront order to scan order synchronization

Systems and methods for generating local image statistics are provided. In one example, an image signal processing system may include a statistics pipeline with image processing logic and local image statistics collection logic. The image processing logic may receive and process pixels of raw image data. The local image statistics collection logic may generate a local histogram associated with a luminance of the pixels of a first block of pixels of the raw image data or a thumbnail in which a pixel of the thumbnail represents a downscaled version of the luminance of the pixels of the first block of the pixel. The raw image data may include many other blocks of pixels of the same size as the first block of pixels.

Local image statistics collection

In one embodiment, a direct memory access (DMA) controller comprises a transmit control circuit, an offload engine, and a receive control circuit. The transmit control circuit is configured to read first DMA data from an address space in a host. Coupled to receive the first DMA data from the transmit control circuit, the offload engine is configured to perform at least a first operation on the first DMA data to produce a result. The offload engine is configured to at least start performing the first operation during a DMA transfer that provides the first DMA data to the offload engine. Coupled to the offload engine to receive the result, the receive control circuit is configured to write the result to the address space in the host according to a DMA descriptor data structure that describes the DMA transfer.

Growing community of inventors

San Jose, CA, United States of America

Weichun Ku