Pixelworks, Inc.

Bob Zhang

Neil Woodall

Yue Ma

Junhua Chen

Guohua Cheng

Guodong Liu

Xiaojie Sheng

Minghui Yang

Tao Chen

James H Zhou

Xuan Wu

Yanxin Li

Qiang Li

Miao Sima

Jian Zhou

Lihang Zhang

Ji Zhao

Richard Miller

Yongmin Hou

Jiehua Shen

Songsong Chen

Jian (James) Zhou

Various methods and systems are provided for image processing for multiple cameras. In one embodiment, a method comprises acquiring image frames with a plurality of image frame sources configured with different acquisition settings, processing the image frames based on the different acquisition settings to generate at least one final image frame, and outputting the at least one final image frame. In this way, information from different image frame sources such as cameras may be leveraged to achieve increased frame rates with improved image quality and a desired motion appearance.

Methods and systems for image processing with multiple image sources

A method includes receiving, at a processor, at least one frame of input image data, producing motion vector fields between the frames of input image data, and applying temporal stability to the at least one frame of the input image data to produce noise reduced image data, wherein applying temporal stability comprises separating pixel data into frequency bands.

Temporal stability for single frame super resolution

A method of converting an input frame rate of input video frames includes upsampling at least some of the input video frames to produce upsampled frames at a higher frame rate than the input frame rate, applying a filter to combine together multiple frames at a single pixel location, downsampling the upsampled frames to an output frame rate. A method of converting an input frame rate of input video frames includes grouping input video frames together, wherein a number of groups per second is less than a capture frame rate, and interpolating the frames within a group to obtain a desired shutter angle, wherein an average time between frames within a group is less than a time between groups.

Realistic 24 frames per second output from high frame rate content

A method of producing video data includes receiving, at a processor, a current frame of image data in a stream of frames of image data, dividing a current frame of image data into blocks, identifying a current block and defining a neighborhood of blocks for the current block, generating at least one initial motion vector for each block, using the initial motion vector for current block and an initial motion model to calculate a weight for each initial motion vector in the neighborhood based on a difference between initial motion vector for the current block and the initial motion vector for at least one other block from the current block in the neighborhood and differences in the image data between the current block and the other blocks in the neighborhood, using the weights for each initial motion vector to generate coefficients for a refined motion model, refining the initial motion vector for the current block according to the refined motion model to produce a refined motion vector, using the refined motion vector and the pixels in the stream of frames to produce at least one of adjusted pixels and new pixels, and displaying the at least one of adjusted pixels and new pixels on a display.

Accuracy and local smoothness of motion vector fields using motion-model fitting

A method of adjusting motion in a video sequence, includes performing motion estimation on an original video sequence and producing initial motion vectors and motion statistics, using the motion statistics to determine a level of motion estimation and motion compensation (MEMC) in the video sequence, adjusting a frame interpolation process based upon the level, and producing a frame interpolated video sequence from the original video sequence and the adjusted frame interpolation process.

Adjusting interpolation phase for motion estimation motion compensation using image analysis

A method of performing motion vector correction in a sequence of video frames includes receiving, at a processor, a frame of video frames at a received rate lower than an original frame rate, identifying motion vector candidates for a frame in the sequence of video frames, detecting a cadence of the sequence of video frames using the motion vector candidates, scaling the motion vector candidates according to the cadence to produce scaled motion vector candidates, calculating motion vectors for a frame in the sequence of video frames using the scaled motion vector candidates, and interpolating at least one new frame of video data using the motion vectors.

Time stamp recovery and frame interpolation for frame rate source

A method includes determining a display backlight level based upon an ambient light level, using image content of incoming image data to adjust the display backlight level based upon image content to produce an image-compensated backlight value, and adjusting pixel values in the image data as needed based upon the image-compensated backlight value.

LCD image compensation for LED backlighting

A method of converting three dimensional image data into two dimensional image data, includes identifying at least two vertices of an object to be rendered in a frame of three dimensional image data, calculating a three-dimensional (3D) motion vector for each vertex of the object to be rendered, determining a position of each vertex in a new frame, calculating the motion vectors for a block based upon the vertex position in the new frame and the motion vectors for the vertex, and using the motion vectors for the vertex to render pixels in the new frame.

2D motion vectors from 3D model data

A system can include a memory and a motion vector calculator (MVC) configured to receive a block from the memory, the MVC including an MVC core configured to generate a partial motion vector (MV) for the block, a halo reduction block configured to perform halo reduction on the partial MV, and an MV post-processing block configured to smooth the partial MV and output the smoothed partial MV.

Frame rate conversion with partial motion vector

A method includes receiving, at a processor, a frame of input image data, generating at least one motion vector field between a first frame of input image data and an adjacent frame of input image data, refining the at least one motion vector field to provide refined motion vectors, temporally filtering a frame of input image data to produce a temporally filtered frame of image data, and generating a frame of interpolated image data using the temporally filtered frame of image data and the refined motion vectors.

A method of developing an image training library includes receiving, at a processor, a set of high resolution image samples at a first resolution, generating a set of low resolution image samples having a second resolution from the set of high resolution images, wherein the second resolution is lower than the first resolution, clustering the low resolution image samples using features in the low resolution images, generating a low resolution dictionary for each cluster, generating sparse coefficients for each sample, and using the sparse coefficients to generate a high resolution dictionary for each cluster.

Image super resolution via spare representation of multi-class sequential and joint dictionaries

A system comprises a phase plane correlation (PPC) processing module configured to receive video data from a video source and generate a motion vector (MV) candidate, and a three-dimensional (3D) recursive processing module configured to receive the MV candidate from the PPC processing module, perform 3D recursive processing on a number of MV candidates including the MV candidate received from the PPC processing module, and select one of the MV candidates based on the 3D recursive processing.

Fusion of phase plane correlation and 3D recursive motion vectors

A method of compensating for match errors while generating a super-resolution image, includes receiving, at a processor, an input image having a first resolution forming a low resolution layer, identifying a patch in a unknown high resolution image layer, identifying a corresponding low resolution patch in the low resolution layer, finding at least one matched patch in a match layer having a resolution lower than the first resolution, identifying at least one copy patch in a copy layer having a resolution lower than the high resolution layer, the copy patch to be copied to the high resolution layer, determining an match error between the at least one matched patch and a corresponding low resolution patch, selecting a matrix to transform the match error to provide compensation for the high resolution patch as match error compensation, and modifying the high resolution patch using the match error compensation.

Match error compensation for super-resolution images

A method of converting two-dimensional image data to three-dimensional image data includes dividing the image data into blocks, performing motion estimation on the blocks to produce block-based motion vectors, applying a global motion analysis and a local motion analysis to the block-based motion vectors to generate motion-based depth, applying a global image model and a local image model to the block-based motion vectors to generate image-based depth, and generating a three-dimensional view by fusing the motion-based depth and the image-based depth. Other conversion methods are also included.

Two dimensional to three dimensional video conversion

An apparatus has a neighbor motion vector module arranged to receive motion vector data for blocks that neighbor a current pixel and select a predetermined number of block motion vectors, at least two pixel motion vector modules arranged to receive the block motion vector data and to produce interpolated pixel motion vector data, a weighting module configured to produce a weighting for the interpolated pixel motion vector data from the motion vector modules, and a blending module to blend the interpolated pixel motion vector data according to the weighting and to produce pixel output data for the current pixel.

Growing community of inventors

Santa Clara, CA, United States of America

Bob Zhang