Ambarella, Inc.

Ambarella International LP

Other

Peter Verplaetse

Leslie D Kohn

Ellen M Lee

Sameer M Gauria

Wen Wan Yang

An apparatus having a processor and a circuit is disclosed. The processor may be configured to (i) compare, at a first level of a motion estimation hierarchy, first units of a current picture with a reference picture to generate first metrics, (ii) combine, at the first level, the first metrics to generate second metrics and (iii) refine, at a second level of the hierarchy, the first metrics and the second metrics to generate motion vectors. Multiple metrics may be refined in parallel. The first metrics generally correspond to the first units in an overlapping unit of the current picture. The second metrics generally correspond to a plurality of second units in the overlapping unit. Each second unit may overlap one or more first units. The circuit may be configured to process the overlapping unit based on the motion vectors to generate an output signal.

Shared motion estimation cost metrics for overlapping units

An apparatus includes a memory and a hardware pipeline. The memory may be configured to store video data. The video data includes a plurality of sections of one or more pictures that may be processed independently. The hardware pipeline comprises a plurality of pipeline stages implementing a video coding process comprising a number of steps. Each of the plurality of pipeline stages performs an associated task of a different step of the video coding process in a substantially similar time on a different one of the plurality of sections as each of the plurality of sections pass through each of the pipeline stages. At least one of the plurality of pipeline stages communicates predictor information that is based on actual neighbor data to an earlier stage of the hardware pipeline.

Interleaved video coding pipeline

An apparatus includes a processor and an operator readiness circuit. The processor may be configured to schedule one or more operators used to process a plurality of vectors based on one or more status signals indicating a readiness state of the one or more operators. The operator readiness circuit may be configured to (i) compare a target position and an actual position of each operand associated with the one or more operators to determine a readiness state of each operand, (ii) update the readiness state of the operands using a plurality of state machines, and (iii) generate the one or more status signals indicating the readiness state of the one or more operators based on the readiness state of each operand associated with the one or more operators.

Scheduler for vector processing operator readiness

An apparatus includes a scheduler circuit and an operator allocation circuit. The scheduler circuit may be configured to (i) parse a directed acyclic graph into one or more operators, (ii) track a plurality of first status signals indicating a readiness state of a plurality of unscheduled operators that are to be allocated to a plurality of hardware engines, (iii) track a plurality of second status signals indicating a readiness state of the hardware engines, and (iv) for each operator, track a resource type parameter. The operator allocation circuit may be configured to (a) select a resource type from a list of resource types in use, (b) determine available hardware engines corresponding to the selected resource type based on the second status signals, (c) generate scores for the unscheduled operators based on (i) the selected resource type, (ii) the first status signals, and (iii) the resource type parameters, and (d) allocate at least one of the unscheduled operators to at least one of the available hardware engines based on the scores.

Scheduler for vector processing operator allocation

An apparatus includes a memory and a circuit coupled to the memory. The memory may be configured as a local buffer to store all or a portion of a first array of values and all or a portion of a second array of values. The circuit may be configured to (i) calculate an intermediate array of values by multiplying a converted version of the first array by a converted version of the second array and (ii) calculate an output array comprising a plurality of output values based on values of the intermediate array and a predefined dimensional reduction.

Generalized dot product for computer vision applications

An apparatus includes a memory and a circuit. The memory may be configured to store a first array of data values, a second array of first modification values, and a third array of second modification values. The circuit may be configured to (a) calculate a plurality of window values for each window position of a first sliding window as the first sliding window is stepped by a predetermined step size along a particular axis of the first array of data values, and (b) calculate the first and the second modification values by summing portions of the first array of data values that correspond to the predetermined step size of the first sliding window. A first window value of an initial position of the first sliding window may be calculated by summing portions of the first array of data values encompassed by the first sliding window at the initial position, and each remaining window value of the plurality of window values may be calculated by using one of the first modification values and one of the second modification values corresponding to a current position of the first sliding window to modify the window value corresponding to a preceding sliding window position.

Computer vision implementing calculations of window sums in overlapping windows

An apparatus includes a memory, a first buffer, a second buffer, and a processing circuit. The memory may be configured to store data. The first buffer may be configured to store a plurality of kernel values fetched from the memory and present a first signal communicating the kernel values as stored. The second buffer may be configured to store a plurality of input tiles fetched from the memory and present a second signal communicating the input tiles as stored. The processing circuit may be configured to (i) receive the first signal and the second signal, (ii) calculate a plurality of intermediate values in parallel by multiplying the input tiles with a corresponding one of the kernel values, and (iii) calculate an output tile comprising a plurality of output values based on the intermediate values. The kernel values are generally fetched from the memory to the first buffer slower than the input tiles are fetched from the memory to the second buffer.

Convolution calculations in multiple dimensions

An apparatus including a memory and a circuit. The memory may be configured to store a multidimensional array of data values. The circuit may be configured to (i) fetch a plurality of data vectors from the memory, where each of the data vectors comprises a plurality of the data values, (ii) calculate a plurality of modification values based on the data values, (iii) calculate a first value of a first window based on the data values, and (iv) calculate a second value of a second window by adding to the first value of the first window a next one of the modification values and subtracting from the first value of the first window a previous one of the modification values. The second window generally overlaps the first window in the multidimensional array along a particular axis.

An apparatus includes a plurality of hardware engines and a scheduler circuit. The hardware engines may be configured to process a plurality of vectors using a plurality of operators. The scheduler circuit may be configured to (i) parse a directed acyclic graph into one or more of the operators, (ii) determine a readiness of each of the operators and (iii) schedule the one or more operators in at least one of the hardware engines based on the readiness. The scheduler circuit may be implemented solely in hardware.

An apparatus includes a plurality of hardware engines and a scheduler circuit. The hardware engines may be configured to process a plurality of vectors using a plurality of operators. The scheduler circuit may be configured to (i) parse a directed acyclic graph into one or more of the operators, (ii) track a plurality of unscheduled operators that have not been allocated to the hardware engines, (iii) track a plurality of statuses of the hardware engines and (iv) allocate at least one of the unscheduled operators to at least one of the hardware engines based on the statuses. The at least one unscheduled operator may be processed in the at least one hardware engine. The scheduler circuit may be implemented solely in hardware.

An apparatus includes a memory and a circuit. The memory may be configured to store data. The circuit generally includes a local buffer. The circuit may be configured to (i) fetch all or a portion of a first array of values from the memory to the local buffer, (ii) fetch all or a portion of a second array of values from the memory to the local buffer, (iii) calculate an intermediate array of values by multiplying a converted version of the first array by a converted version of the second array, and (iv) calculate an output array comprising a plurality of output values based on values of the intermediate array and a predefined dimensional reduction.

An apparatus includes a memory and a circuit. The memory may be configured to store data. The circuit generally has a buffer and may be configured to (i) fetch a kernel from the memory, where the kernel may have a plurality of kernel values, (ii) fetch a block from the memory to the buffer, where the block may have a plurality of input tiles and each of the input tiles may have a plurality of input values in multiple dimensions, (iii) calculate a plurality of intermediate values in parallel by multiplying the input tiles read from the buffer with a corresponding one of the kernel values and (iv) calculate an output tile that may have a plurality of output values based on the intermediate values.

An apparatus includes a first circuit and a second circuit. The first circuit may be configured to (i) fetch a reference samples from a memory to slots in a buffer, (ii) generate motion vectors by motion estimating inter-prediction candidates of a current picture relative to the reference samples in the buffer, (iii) snoop the fetches from the memory to determine if the reference samples fetched for a non-zero motion vector type of the inter-prediction candidates includes the reference samples for a zero motion vector type of the inter-prediction candidates and (iv) avoid duplication of the fetches for the zero motion vector type of the inter-prediction candidates where the snoop determines that the reference samples have already been fetched. The second circuit may be configured to evaluate the reference samples in the buffer based on the motion vectors to select a prediction sample unit made of the reference samples.

Inter-prediction candidate selection in a mode decision

An apparatus includes a circuit and a processor. The circuit may be configured to (i) generate a plurality of sets of coefficients by compressing a tile in a picture in a video signal at each of a plurality of different sizes of a plurality of coding units in a coding tree unit and (ii) reconstruct the tile based on a particular one of the sets of coefficients. The sets of coefficients may be generated at two or more of the different sizes of the coding units in parallel. Each of the sets of coefficients may be generated in a corresponding one of a plurality of pipelines that operate in parallel. Each of the sets of coefficients may have a same number of the coefficients. The processor may be configured to select the particular set of coefficients in response to the compression of the tile.

Tree-coded video compression with coupled pipelines

An apparatus includes a memory and a processor. The memory may be configured to store video data. The video data includes a plurality of sections of one or more pictures that can be processed independently. The processor generally includes a hardware pipeline. The hardware pipeline implements a number of stages of a video coding process, such that each stage performs an associated task in a substantially similar time on a different one of said plurality of sections.

Growing community of inventors

Redwood City, CA, United States of America

Peter Verplaetse