Sony Corporation

Canon Kabushiki Kaisha

Sony Electronics Inc

Sony Group Corporation

Sony Corporation of America

Alexander Berestov

Chuen-Chien Lee

Cheng-Yi Liu

Jianing Wei

Xue Tu

Florian Ciurea

Farhan A Baqai

Ted J Cooper

Kenichi Nishio

Xiaoling Wang

Naoya Katoh

Pingshan Li

Golnaz Abdollahian

Hans Masli

Masaya Kinoshita

Hiroshige Okamoto

Takami Mizukura

Jeongnam Youn

Hiromasa Naganuma

Harry T Garland

Gazi Ali

Jiro Takatori

Ivan Bojer

Houman Rastgar

Ting Zhang

Soo Hyun Bae

Autumn Claire Spaulding

Ali Jezu Tabatabai

Ming-Chang Liu

Akira Matsui

Jun Yokono

Roger D Melen

Mohammad Gharavi-Alkhansari

Seijiro Inaba

Ryota Kosakai

Daisuke Tahara

Earl Quong Wong

Hiroyuki Takeda

Daniel A Usikov

Xiaogang Dong

Tatsuya Deguchi

Wei Ji

Tomonori Shuda

Shohei Noguchi

Hiroaki Ebi

Tak Shing Wong

Jianfei Max Chen

Tomofumi Okuda

Jeremy Saltzman

Owen Mayer

Somayeh Bakhtiari

Daniel Usikov

A method for optimization of alignment of pixels of X different types on a sensor array for the performance of N different camera tasks, where X and N are integers greater than 1, includes obtaining an output of each of the X types of pixel on the sensor array for an input image; evaluating the quality of the output for each of the N different camera tasks; and obtaining an optimal pixel alignment pattern by adjustment of a potential pixel alignment pattern with respect to performance of the N different camera tasks subject to a user-determined balance of priorities among the N different camera tasks.

Method for automatic sensor pixel arrangement optimized for multiple camera tasks

A method of 3D motion reconstruction of outdoor, highly flexible moving subjects with multiple cameras on drones is described herein. A multi-drone capturing system is used to remove the restriction of a dedicated virtual reality shooting place, 'the hot seat,' so that the actor/target is able to perform agile or long-distance activities outdoor. The subject's 3D pose parameters are estimated by the captured multi-view images by the drones, and the sequence of poses becomes the motion in time to control the animation of a pre-built 3D model. The method is able to be directly integrated into an existing Virtual Reality/Augmented reality (VR/AR) production chain, and the subject is able to be extended to animals that are difficult to be contained in a VR camera mesh space.

Method of 3D reconstruction of dynamic objects by mobile cameras

A method of camera control for a camera capturing images of a target includes a sequence of at least four steps. The first step determines in real time a location of a drone and a pose of a camera on the drone. The second step, which may occur before or after the first step, determines in real time a location of a reference object, the location of the reference object having a fixed relationship to a location of the target. The third step uses the determined locations to calculate a distance, characterized by magnitude and direction, between the target and the drone. The fourth step uses the calculated distance to control the pose of the camera such that an image captured by the camera includes the target. Controlling the pose of the camera does not require any analysis of the captured image.

Method of multi-drone camera control

A system and method for calibration of cameras on Unmanned Aerial Vehicles (UAVs) is provided. The system receives a set of anchor images of a human subject from a set of anchor cameras and a group of images of the human subject from multiple viewpoints in three-dimensional (3D) space from a group of cameras on a group of UAVs. The system determines a first set of two-dimensional (2D) positions of human joints from the set of anchor images and a second set of 2D positions of the human joints from the group of images. The system computes, as 3D key-points, 3D positions of the human joints based on triangulation using the first set of 2D positions and determines a 2D re-projection error between the 3D key-points and the second set of 2D positions. Thereafter, by minimizing the 2D re-projection error, the system calibrates each camera of the group of cameras.

Calibration of cameras on unmanned aerial vehicles using human joints

Various aspects of a system and a method for camera calibration by use of a rotatable three-dimensional (3-D) calibration object are disclosed herein. In accordance with an embodiment, the system includes a first electronic device, which determines a rotation pattern of the 3-D calibration object, based on a set of pre-selected images. The set of pre-selected images includes the 3-D calibration object captured at pre-defined viewing angles. Control information is communicated by the first electronic device to a second electronic device associated with the 3-D calibration object to rotate the 3-D calibration object in accordance with the determined rotation pattern. A plurality of image frames of the 3-D calibration object are captured to calibrate intrinsic and/or extrinsic camera parameters of the first electronic device.

System and method for camera calibration by use of rotatable three-dimensional calibration object

Various aspects of a system and method for utilization of multiple-camera network to capture static and/or motion scenes are disclosed herein. The system comprises a plurality of unmanned aerial vehicles (UAVs). Each of the plurality of UAVs is associated with an imaging device configured to capture a plurality of images. A first UAV of the plurality of UAVs comprises a first imaging device configured to capture a first set of images of one or more static and/or moving objects. The first UAV is configured to receive focal lens information, current location and current orientation from one or more imaging devices. A target location and a target orientation of each of the one or more imaging devices are determined. Control information is communicated to the one or more other UAVs to modify the current location and current orientation to the determined target location and orientation of each of the one or more imaging devices.

System and method for utilization of multiple-camera network to capture static and/or motion scenes

Various aspects of a method and a system for image processing are disclosed herein. The method includes processing an input image, which comprises structure information and detail information, using image processing (IP) blocks in an image processing pipeline. One or more of the IP blocks, such as the 'lossy' IP blocks, process the input image with at least a partial loss of the detail information. By replacing the 'lossy' IP blocks with redesigned image processing (IP) modules, the image processing pipeline reduces or avoids such loss of the detail information. A more efficient implementation of the improved pipeline is realized by using a master IP module when the &#x201c;lossy&#x201d; IP blocks are reordered and grouped together in the image processing pipeline. The method is further extended to process 3-D images to reduce or avoid loss of detail information in a 3-D image processing pipeline.

Method and system for improving detail information in digital images

Various aspects of a system and a method for extrinsic camera parameters calibration by use of a three dimensional (3D) calibration are disclosed herein. In accordance with an embodiment, the system includes an electronic device, which receives an image, which includes the 3D calibration object with a plurality of unique identifiers. One or more appearance parameters of the 3D calibration object and/or a plurality of intrinsic camera parameters of an image-capturing device that captured the received image, are further received. A positional probability is computed for each of the plurality of unique identifiers distributed in the received image. The plurality of unique identifiers of the 3D calibration object are located in the received image to calibrate the extrinsic camera parameters based on the computed positional probability, the received one or more appearance parameters of the 3D calibration object, and the plurality of intrinsic camera parameters.

System and method for extrinsic camera parameters calibration by use of a three dimensional (3D) calibration object

A method to position multiple camera using camera ordering is described herein. Based on two known camera positions, a third camera position is able to be determined using acquired information and calculations.

Method of multiple camera positioning utilizing camera ordering

A method to estimate a set of camera locations, in clockwise or counter-clockwise order, according to the videos captured by these cameras is described herein. In some embodiments, the cameras are assumed to be fixed, with no or very mild tilting angles and no rolling angles (the horizon is horizontal in each camera image). The difference of orientation (rolling angle) between each neighboring (closest) camera pair is able to be up to 45 degrees. Each camera is assumed to have overlapped views with at least one other camera. Each camera has one right neighboring camera and one left neighboring camera, except the first and the last cameras which have only one neighboring camera at one side. The locations of the cameras then are able to be expressed as a unique list counter-clockwise. The input videos are assumed to be synchronized in time.

Method of fast and robust camera location ordering

The thick lens calibration method enables better calibration of complex camera devices such as devices with thick lens systems. The thick lens calibration method includes a two step process of calibrating using the distance between a second nodal point and an image sensor, and calibrating using the distance between the first and second nodal point.

Calibration methods for thick lens model

Various aspects of a method and system to track one or more objects in a video stream are disclosed herein. In accordance with an embodiment, the method includes computation of a first confidence score of a first geometrical shape that encompasses at least a portion of an object in a first image frame of the video stream. The first geometrical shape is utilized to track the object in the video stream. The first geometrical shape is split into a plurality of second geometrical shapes. The split of the first geometrical shape is based on a comparison of the computed first confidence score with a pre-defined threshold score.

Method and system to adaptively track objects

Various aspects of a method and system for object tracking are disclosed herein. The method comprises computation of an overlap value between a first bounding box and a second bounding box, associated with an object in an image frame of a video stream. The overlap values are utilized to track the object in the video stream. The first bounding box encompasses at least a partial first portion of an object. The second bounding box encompasses at least a partial second portion of the object. A first score associated with the first bounding box and a second score associated with the second bounding box is computed based on a comparison of the computed overlap value with a pre-defined threshold value. The first bounding box is updated to a new bounding box that at least partially encompasses the object, based on the computed first score and the computed second score.

Method and system for object tracking

Focus detection is to determine whether an image is in focus or not. Focus detection is able to be used for improving camera autofocus performance. Focus detection by using only one feature does not provide enough reliability to distinguish in-focus and slightly out-of-focus images. A focus detection algorithm of combining multiple features used to evaluate sharpness is described herein. A large image data set with in-focus and out-of-focus images is used to develop the focus detector for separating the in-focus images from out-of-focus images. Many features such as iterative blur estimation, FFT linearity, edge percentage, wavelet energy ratio, improved wavelet energy ratio, Chebyshev moment ratio and chromatic aberration features are able to be used to evaluate sharpness and determine big blur images.

Focus detection

An image processing system and method of operation includes: a source image having source pixels; homogeneous blocks in the source image having a block color; a homogeneous region in the source image formed by merging the homogeneous blocks having the block color within a color threshold; a text background region having text pixels and background pixels in the homogeneous region with the text background region having a texture feature above a texture threshold and a region size above a region size threshold; and a binary text mask representing the text pixels and the background pixels for displaying on a device.

Image processing system for cluttered scenes and method of operation thereof

Growing community of inventors

San Jose, CA, United States of America

Alexander Berestov