Pixelworks, Inc.

Guodong Liu

Junhua Chen

Bob Zhang

Neil Woodall

Yue Ma

Minghui Yang

Qiang Li

Lihang Zhang

Jiehua Shen

A method includes receiving, at a processor, at least one frame of input image data, producing motion vector fields between the frames of input image data, and applying temporal stability to the at least one frame of the input image data to produce noise reduced image data, wherein applying temporal stability comprises separating pixel data into frequency bands.

Temporal stability for single frame super resolution

A method of producing video data includes receiving, at a processor, a current frame of image data in a stream of frames of image data, dividing a current frame of image data into blocks, identifying a current block and defining a neighborhood of blocks for the current block, generating at least one initial motion vector for each block, using the initial motion vector for current block and an initial motion model to calculate a weight for each initial motion vector in the neighborhood based on a difference between initial motion vector for the current block and the initial motion vector for at least one other block from the current block in the neighborhood and differences in the image data between the current block and the other blocks in the neighborhood, using the weights for each initial motion vector to generate coefficients for a refined motion model, refining the initial motion vector for the current block according to the refined motion model to produce a refined motion vector, using the refined motion vector and the pixels in the stream of frames to produce at least one of adjusted pixels and new pixels, and displaying the at least one of adjusted pixels and new pixels on a display.

Accuracy and local smoothness of motion vector fields using motion-model fitting

A method includes receiving, at a processor, a frame of input image data, generating at least one motion vector field between a first frame of input image data and an adjacent frame of input image data, refining the at least one motion vector field to provide refined motion vectors, temporally filtering a frame of input image data to produce a temporally filtered frame of image data, and generating a frame of interpolated image data using the temporally filtered frame of image data and the refined motion vectors.

A method of generating super resolution image data includes receiving original image data of a low resolution at an image processing device, performing motion compensation on the original image data using a current frame of image data and at least one previous frame of image data and at least one future frame of image data as reference frames, generating motion vectors, applying noise reduction to the current frame of image data and the reference frames to produce noise reduced, current frame image data, and generating a current frame of super resolution image data using the noise reduced, current frame image data.

Noise reduction with multi-frame super resolution

A method of developing an image training library includes receiving, at a processor, a set of high resolution image samples at a first resolution, generating a set of low resolution image samples having a second resolution from the set of high resolution images, wherein the second resolution is lower than the first resolution, clustering the low resolution image samples using features in the low resolution images, generating a low resolution dictionary for each cluster, generating sparse coefficients for each sample, and using the sparse coefficients to generate a high resolution dictionary for each cluster.

Image super resolution via spare representation of multi-class sequential and joint dictionaries

A system comprises a phase plane correlation (PPC) processing module configured to receive video data from a video source and generate a motion vector (MV) candidate, and a three-dimensional (3D) recursive processing module configured to receive the MV candidate from the PPC processing module, perform 3D recursive processing on a number of MV candidates including the MV candidate received from the PPC processing module, and select one of the MV candidates based on the 3D recursive processing.

Fusion of phase plane correlation and 3D recursive motion vectors

A method of compensating for match errors while generating a super-resolution image, includes receiving, at a processor, an input image having a first resolution forming a low resolution layer, identifying a patch in a unknown high resolution image layer, identifying a corresponding low resolution patch in the low resolution layer, finding at least one matched patch in a match layer having a resolution lower than the first resolution, identifying at least one copy patch in a copy layer having a resolution lower than the high resolution layer, the copy patch to be copied to the high resolution layer, determining an match error between the at least one matched patch and a corresponding low resolution patch, selecting a matrix to transform the match error to provide compensation for the high resolution patch as match error compensation, and modifying the high resolution patch using the match error compensation.

Match error compensation for super-resolution images

A method of producing a high resolution image includes generating high resolution image data for a high resolution image using a processor, wherein the high resolution image has a resolution higher than a resolution of an original image, determining image content of the high resolution image data, and selectively adding dither noise to the high resolution image data based upon the image content of the high resolution image data to produce final image data.

Visually comfortable content-adaptive dithering for super-resolution

A method of processing video data includes upscaling an input low resolution image to a high resolution image using a processor, detecting monotonicity in a direction normal to an edge in a low resolution neighborhood in the input low resolution image, and correcting pixels in a high resolution neighborhood of the high resolution image corresponding to the low resolution neighborhood to preserve at least some of the monotonicity.

Overshoot protection of upscaled images

A method of generating a super-resolution image from a single frame of image data includes using a processor to retrieve query patches of image data from a memory, determining a search range for each patch, and generating super-resolution image data corresponding to each patch based upon the search range.

Low power patch matching in self-similarity based single frame super-resolution

A method of generating a super-resolution image from a single frame includes receiving, at a processor, first image data having a first resolution as a query image layer, acquiring, by the processor, second image data as a copy image layer, generating, by the processor, at least one additional image layer as a retrieval layer, the at least one additional image layer having a resolution lower than the first resolution, wherein the lower resolution corresponds to the first resolution divided by a scaling factor, generating high resolution image data for a high resolution layer having a resolution higher than the first resolution, the generating including copying image data from the copy layer when a match is found and upscaling the query layer when a match is not found.

Self-similarity based single frame super-resolution

A method can include receiving an input image, performing an edge-preservation enhancement on the input image, performing a local-extrema enhancement on the input image, and performing a digital transient improvement (DTi) operation on the input image. A gradient-based fusion of an output of the edge-preserving enhancement and an output of the local-extrema enhancement may be performed, and a transient-based fusion of an output of the gradient-based fusion and an output of the DTi operation may also be performed.

Image detail enhancement and edge sharpening without overshooting

A method of generating a super-resolution image from a single frame of image data, includes receiving, at a processor, a frame of image data have a first resolution as an original image layer, generating an estimated super-resolution image as a high resolution image layer, the estimated super-resolution image having a second resolution, the second resolution being higher than the first resolution from the frame of original image data, generating a low resolution image layer by down sampling the frame of image data, the low resolution image layer having a resolution lower than the first resolution, down sampling the estimated super-resolution image to produce an intermediate image layer having an intermediate resolution between the first resolution and the second resolution, and selectively copying image data from the original image layer to the super-resolution image layer to produce a final super-resolution image.

Recovering details in single frame super resolution images

A method of performing motion compensation includes dividing at least one frame of image data into blocks, performing phase plane correlation to determine a correlation surface for each block between a first frame and a second frame, using the correlation surfaces for each block in the first frame to produce a global correlation surface for a first frame, using the global correlation surface to produce a refined correlation surface, selecting peaks in the refined correlation surface, and perform sub-pixel motion vector calculations to produce global motion vectors using the peaks.

Growing community of inventors

Shanghai, China

Guodong Liu