Fotonation Limited

Adeia Semiconductor Bonding Technologies Inc.

Mihai Constantin Munteanu

Corneliu Zaharia

Dragos Dinu

Alexandru Caliman

Vlad Georgescu

Alexei Pososin

Piotr Stec

Petronel Bigioi

Szabolcs Fulop

Gyorgy Simon

Iulia Suciu

A method of tracking an object across a stream of images comprises determining a region of interest (ROI) bounding the object in an initial frame of an image stream. A HOG map is provided for the ROI by: dividing the ROI into an array of M×N cells, each cell comprising a plurality of image pixels; and determining a HOG for each of the cells. The HOG map is stored as indicative of the features of the object. Subsequent frames are acquired from the stream of images. The frames are scanned ROI by ROI to identify a candidate ROI having a HOG map best matching the stored HOG map features. If the match meets a threshold, the stored HOG map indicative of the features of the object is updated according to the HOG map for the best matching candidate ROI.

Method and system for tracking an object

A method of tracking an object across a stream of images comprises determining a region of interest (ROI) bounding the object in an initial frame of an image stream. A HOG map is provided for the ROI by: dividing the ROI into an array of M&#xd7;N cells, each cell comprising a plurality of image pixels; and determining a HOG for each of the cells. The HOG map is stored as indicative of the features of the object. Subsequent frames are acquired from the stream of images. The frames are scanned ROI by ROI to identify a candidate ROI having a HOG map best matching the stored HOG map features. If the match meets a threshold, the stored HOG map indicative of the features of the object is updated according to the HOG map for the best matching candidate ROI.

An image processing apparatus comprises a normalisation module operatively connected across a bus to a memory storing an image in which a region of interest (ROI) has been identified within the image. The ROI is bound by a rectangle having a non-orthogonal orientation within the image. In one embodiment, the normalisation module is arranged to divide the ROI into one or more slices, each slice comprising a plurality of adjacent rectangular tiles. For each slice, the apparatus successively reads ROI information for each tile from the memory including: reading a portion of the image extending across at least a width of the slice line-by-line along an extent of a slice. For each tile, the apparatus downsamples the ROI information to a buffer to within a scale S&#x3c;2 of a required scale for a normalised version of the ROI. The apparatus then fractionally downsamples and rotates downsampled information for a tile within the buffer to produce a respective normalised portion of the ROI at the required scale for the normalised ROI. Downsampled and rotated information is accumulated for each tile within a normalised ROI buffer for subsequent processing by the image processing apparatus.

Image processing apparatus

A convolutional neural network (CNN) for an image processing system comprises an image cache responsive to a request to read a block of N&#xd7;M pixels extending from a specified location within an input map to provide a block of N&#xd7;M pixels at an output port. A convolution engine reads blocks of pixels from the output port, combines blocks of pixels with a corresponding set of weights to provide a product, and subjects the product to an activation function to provide an output pixel value. The image cache comprises a plurality of interleaved memories capable of simultaneously providing the N&#xd7;M pixels at the output port in a single clock cycle. A controller provides a set of weights to the convolution engine before processing an input map, causes the convolution engine to scan across the input map by incrementing a specified location for successive blocks of pixels and generates an output map within the image cache by writing output pixel values to successive locations within the image cache.

Convolutional neural network

A method for producing a histogram of oriented gradients (HOG) for at least a portion of an image comprises dividing the image portion into cells, each cell comprising a plurality of image pixels. Then, for each image pixel of a cell, obtaining a horizontal gradient component, gx, and a vertical gradient component, gy, based on differences in pixel values along at least a row of the image and a column of the image respectively including the pixel; and allocating a gradient to one of a plurality of sectors, where n is a sector index, each sector extending through a range of orientation angles and at least some of the sectors being divided from adjacent sectors according to the inequalities: b*16&#x3c;a*(n+1) where 0&#x2264;n&#x2264;5, b=gy and a=gx; and b*16&#x3c;a*(13&#x2212;n) where 7&#x2264;n&#x2264;12, a=gx+gy and b=gx&#x2212;gy. At least one sector is associated with a bin; and each instance of a pixel gradient of a cell associated with a bin is accumulated to provide a HOG for the cell.

Method for producing a histogram of oriented gradients

A neural network engine comprises a plurality of floating point multipliers, each having an input connected to an input map value and an input connected to a corresponding kernel value. Pairs of multipliers provide outputs to a tree of nodes, each node of the tree being configured to provide a floating point output corresponding to either: a larger of the inputs of the node; or a sum of the inputs, one output node of the tree providing a first input of an output module, and one of the multipliers providing an output to a second input of the output module. The engine is configured to process either a convolution layer of a neural network, an average pooling layer or a max pooling layer according to the kernel values and whether the nodes and output module are configured to output a larger or a sum of their inputs.

Neural network engine

A method for producing a histogram of oriented gradients (HOG) for at least a portion of an image comprises dividing said image portion into cells, each cell comprising a plurality of image pixels. For each image pixel of a cell, a horizontal gradient component, gx, and a vertical gradient component, gy, is obtained based on differences in pixel values along at least a row of said image and a column of said image respectively including the pixel. A gradient is allocated to one of a plurality of sectors, each sector extending through a range of orientation angles. At least some of said sectors are divided from adjacent sectors along lines including gx=2&#xb7;gy, where n is any integer value with a magnitude greater than or equal to 1. At least one sector is associated with a bin; and a count of each instance of a pixel gradient of a cell associated with a bin is performed to provide a HOG for said cell.

A method for correcting a distorted input image comprises determining a local region of an image to be displayed and dividing said region into an array of rectangular tiles, each tile corresponding to a distorted tile with a non-rectangular boundary within said input image. For each tile of the local region, maximum and minimum memory address locations of successive rows of said input image sufficient to span said boundary of said distorted tile are determined. Successive rows of the distorted input from between said maximum and minimum addresses are read. Distortion of the non-rectangular portion of said distorted input image is corrected to provide a tile of a corrected output image which is stored.

Method and system for correcting a distorted input image

A method for correcting a distorted input image includes determining a local region of an image to be displayed and dividing the region into an array of rectangular tiles, each tile corresponding to a distorted tile with a non-rectangular boundary within the input image. For each tile of the local region, maximum and minimum memory address locations of successive rows of the input image sufficient to span the boundary of the distorted tile are determined. Successive rows of the distorted input from between the maximum and minimum addresses are read. Distortion of the non-rectangular portion of the distorted input image is corrected to provide a tile of a corrected output image which is stored.

Growing community of inventors

Brasov, Romania

Mihai Constantin Munteanu