Sony Corporation

Sony Electronics Inc

Sony Group Corporation

Pingshan Li

Kensuke Miyagi

Earl Quong Wong

Florian Ciurea

Soroj Triteyaprasert

Yoshihiro Murakami

Hidenori Kushida

Mamoru Sugiura

Tomonori Shuda

Akira Matsui

Alexander Berestov

Takami Mizukura

Makibi Nakamura

Gazi Ali

Xue Tu

Jianing Wei

Eric D Edwards

Masato Watanabe

Maikibi Nakamura

Yi Sun

Somayeh Bakhtiari

An image processing apparatus and method for detecting and tracking multiple moving objects in successive frames includes circuitry configured to detect a plurality of moving regions within a plurality of frames based on an optical flow map of a plurality of pixels within the plurality of moving regions. A plurality of merged objects are identified from the detected plurality of moving regions. A plurality of split objects are identified from the detected plurality of moving regions and the identified plurality of merged objects. A location of each of the detected plurality of moving regions, the identified plurality of merged objects and the identified plurality of split objects is predicted and tracked within the captured plurality of frames.

Optical flow based detection and tracking of multiple moving objects in successive frames

An apparatus and method for detection of moving objects in a sequence of frames, includes circuitry, a memory, and an image-capture device. The circuitry derives a first optical flow map based on motion information of a first frame and a second frame of a plurality of frames. A second optical flow map is further derived, based on motion information of the second frame and a third frame of the plurality of frames. A first foreground region is further identified that corresponds to the moving objects across the first frame and the second frame and the identified first foreground region is warped across the first frame and the second frame and the moving objects are detected, based on combination of a plurality of pixels in the warped first foreground region and the second foreground region.

Moving object detection in image frames based on optical flow maps

A method and a device, such as an imaging device, for image processing for automatic detection of focus area are disclosed herein. In accordance with an embodiment, the device for image processing is the imaging device that is configured to extract a plurality of object features of a plurality of objects in a field-of-view (FOV) of the imaging device. A plurality of confidence maps are generated based on the extracted plurality of object features. The imaging device is further configured to determine a focus area corresponding to the FOV, based on the generated plurality of confidence maps by use of a pre-determined rule.

Image processing for automatic detection of focus area

Various aspects of a system and method for video processing is disclosed herein. The system comprises a video processing device that is configured to generate a spatial saliency map based on spatial information associated with a current frame of a video stream. A spatio-temporal saliency map is generated based on at least motion information associated with the current frame and a previous frame of the video stream. Based on a weighted combination of the generated spatial saliency map and the generated spatio-temporal saliency map, a combined saliency map is generated.

Method and system for video processing

Various aspects of a device and method for detection of regions based on sensor data for autofocus are disclosed herein. In accordance with an embodiment, the method includes receipt of a depth of filed (DOF) information of a scene viewed using an image-capturing device of the electronic device. The one or more images of the scene are segmented into a plurality of candidate segmented regions based on the received DOF information. One of the plurality of candidate segmented regions is determined as a region-of-interest to autofocus on the determined candidate segmented region.

Device and method for detecting regions in an image

Various aspects of a system and a method for background subtraction in video content are disclosed herein. The system includes an electronic device, which generates an optical flow image of an image frame from a sequence of image frames. A first distribution is determined based on pixel values of a plurality of pixels of the generated optical flow image. A second distribution is determined based on pixel values of a set of pixels of the plurality of pixels of the generated optical flow image. The set of pixels correspond to pixels that touch one or more borders of the image frame. A third distribution is further determined based on the determined first distribution and the determined second distribution. One or more peak values corresponding to one or more background regions in the generated optical flow image, are determined within a specified threshold range in the determined third distribution.

System and method for background subtraction in video content

The thick lens calibration method enables better calibration of complex camera devices such as devices with thick lens systems. The thick lens calibration method includes a two step process of calibrating using the distance between a second nodal point and an image sensor, and calibrating using the distance between the first and second nodal point.

Calibration methods for thick lens model

A blur estimation method and apparatus which utilizes Gaussian differences at multiple resolutions, upon which iterative convolution is performed to match sharpness measures between two images. Blur differences are combined with an iteration number selected from multiple resolution levels based on comparing a sharpness ratio as the blur difference estimate. Blur difference estimate is then applied through a depth model to arrive at a depth estimate. Blur difference depth estimation can be utilized for controlling operation of focus control hardware in an imaging device.

Depth estimation from image defocus using multiple resolution Gaussian difference

Focus detection is to determine whether an image is in focus or not. Focus detection is able to be used for improving camera autofocus performance. Focus detection by using only one feature does not provide enough reliability to distinguish in-focus and slightly out-of-focus images. A focus detection algorithm of combining multiple features used to evaluate sharpness is described herein. A large image data set with in-focus and out-of-focus images is used to develop the focus detector for separating the in-focus images from out-of-focus images. Many features such as iterative blur estimation, FFT linearity, edge percentage, wavelet energy ratio, improved wavelet energy ratio, Chebyshev moment ratio and chromatic aberration features are able to be used to evaluate sharpness and determine big blur images.

Focus detection

A defocus estimation algorithm is described herein. The defocus estimation algorithm utilizes a single image. A Laplacian of Gaussian approximation is determined by computing a difference of Gaussian. Defocus blur is able to be estimated by computing a blur difference between two images using the difference of Gaussian.

Defocus estimation from single image based on Laplacian of Gaussian approximation

A system and method of operation of an image processing system includes: a capture image sequence module for receiving an image sequence; a calculate iteration depth map module for calculating an iteration depth map and a zero-crossing array; a calculate median depth map module for calculating a median depth map and a depth difference map; a calculate variance depth map module for calculating a variance depth map; a calculate image depth map module for calculating an image depth map based on the iteration depth map, the zero-crossing array, the median depth map, the depth difference map, and the variance depth map; and a calculate display image module for calculating a display image based on the received images and the image depth map for displaying on a display device.

Image processing system with hybrid depth estimation and method of operation thereof

A system and method of operation of an image processing system includes: a receive images module for receiving a first image having a first aperture and a second image having a second aperture with the first image and the second image each having a red channel and a green channel; a calculate blur difference module for calculating a red iteration count of a blur difference for the red channel of the first image and the second image, and calculating a green iteration count of the blur difference for the green channel of the first image and the second image; a calculate depth map module for forming a depth map having an element depth based on the red iteration count and the green iteration count; and a calculate display image module for forming a display image based on the first image and the depth map for displaying on a display device.

Image processing system with depth map determination based on iteration count of blur difference and method of operation thereof

A system and method of operation of an imaging system includes: a capture module for receiving an input sequence with an image pair, the image pair having a first image and a second image and for determining a first patch from the first image and a second patch from the second image; an iteration module for generating an iteration map from the first patch of the first image and the second patch of the second image; and a position module for generating a final depth map from the iteration map and for determining a focus position from the final depth map.

Imaging system with depth estimation mechanism and method of operation thereof

A methodology for performing a depth estimation procedure with defocused images under extreme lighting conditions includes a camera device with a sensor for capturing blur images of a photographic target under extreme lighting conditions. The extreme lighting conditions may include over-exposed conditions and/or under-exposed conditions. The camera device also includes a depth generator that performs the depth estimation procedure by utilizing the captured blur images. The depth estimation procedure includes a clipped-pixel substitution procedure to compensate for the extreme lighting conditions.

Methodology for performing depth estimation with defocused images under extreme lighting conditions

A system and method for supporting a depth estimation procedure by utilizing an adaptive kernel includes a capture subsystem for capturing images of a photographic target. The capture subsystem includes an aperture that is adjustable for admitting reflected light from the photographic target to a sensor device. An adaptive kernel is designed in a kernel design procedure based upon symmetry characteristics of the aperture. The adaptive kernel may be designed in either a frequency-domain kernel design procedure or in a spatial-domain kernel design procedure. A depth estimator utilizes the adaptive kernel for performing the depth estimation procedure.

Growing community of inventors

Sunnyvale, CA, United States of America

Pingshan Li