Sony Corporation

Google Inc.

Sony Group Corporation

Xue Tu

Chia-Kai Liang

Jianing Wei

Alexander Berestov

Fuhao Shi

Pingshan Li

Xiaoling Wang

Lun-Cheng Chu

Akira Matsui

Takami Mizukura

Chuen-Chien Lee

Tomonori Shuda

Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for video stabilization. In some implementations, a computer system obtains frames of a video captured by a recording device using an optical image stabilization (OIS) system. The computing system receives (i) OIS position data indicating positions of the OIS system during capture of the frames, and (ii) device position data indicating positions of the recording device during capture of the frames. The computing system determines a first transformation for a particular frame based on the OIS position data for the particular frame and device position data for the particular frame. The computing system determines a second transformation for the particular frame based on the first transformation and positions of the recording device occurring after capture of the particular frame. The computing system generates a stabilized version of the particular frame using the second transformation.

Video stabilization

The disclosed technology includes techniques for providing improved video stabilization on a mobile device. Using gyroscope data of the mobile device, the physical camera orientation of the mobile device may be estimated over time. Using the physical camera orientation and historical data, corresponding virtual camera orientations representing a camera orientation with undesired rotational movement removed may be modeled using a non-linear filter to provide for mapping of a real image to a stabilized virtual image. The virtual camera orientation may be modified to prevent undefined pixels from appearing in the output image.

Video stabilization for mobile devices

The thick lens calibration method enables better calibration of complex camera devices such as devices with thick lens systems. The thick lens calibration method includes a two step process of calibrating using the distance between a second nodal point and an image sensor, and calibrating using the distance between the first and second nodal point.

Calibration methods for thick lens model

A blur estimation method and apparatus which utilizes Gaussian differences at multiple resolutions, upon which iterative convolution is performed to match sharpness measures between two images. Blur differences are combined with an iteration number selected from multiple resolution levels based on comparing a sharpness ratio as the blur difference estimate. Blur difference estimate is then applied through a depth model to arrive at a depth estimate. Blur difference depth estimation can be utilized for controlling operation of focus control hardware in an imaging device.

Depth estimation from image defocus using multiple resolution Gaussian difference

Focus detection is to determine whether an image is in focus or not. Focus detection is able to be used for improving camera autofocus performance. Focus detection by using only one feature does not provide enough reliability to distinguish in-focus and slightly out-of-focus images. A focus detection algorithm of combining multiple features used to evaluate sharpness is described herein. A large image data set with in-focus and out-of-focus images is used to develop the focus detector for separating the in-focus images from out-of-focus images. Many features such as iterative blur estimation, FFT linearity, edge percentage, wavelet energy ratio, improved wavelet energy ratio, Chebyshev moment ratio and chromatic aberration features are able to be used to evaluate sharpness and determine big blur images.

Focus detection

An image processing system and method of operation includes: an image processing device; an image capture module for capturing a source image on the image processing device; an assessment module for detecting an edge of the source image, and measuring an edge width measure of the edge; and a retrieving module for retrieving a depth of field from the edge width measure.

Image processing system with blur measurement and method of operation thereof

A method for converting a 2D image into a 3D image includes receiving the 2D image; determining whether the received 2D image is a portrait, wherein the portrait can be a face portrait or a non-face portrait; if the received 2D image is determined to be a portrait, creating a disparity between a left eye image and a right eye image based on a local gradient and a spatial location; generating the 3D image based on the created disparity; and outputting the generated 3D image.

Methods and systems for 2D to 3D conversion from a portrait image

A method for converting a 2D image into a 3D image includes receiving the 2D image; analyzing content of the received 2D image; determining a 2D-to-3D image conversion method based on a result of the content analysis; generating the 3D image by applying the determined method to the received 2D image; and providing the generated 3D image.

2D to 3D image conversion based on image content

A method for generating a depth map for a 2D image includes receiving the 2D image; analyzing content of the received 2D image; determining a depth map based on a result of the content analysis; refining the determined depth map using an edge-preserving and noise reducing smoothing filter; and providing the refined depth map.

Growing community of inventors

Fremont, CA, United States of America

Xue Tu