Sony Group Corporation

Sony Corporation

Sony Semiconductor Solutions Corporation

Sony Europe B.v.

Francesco Michielin

Fredrik Mattisson

Lars Novak

Fredrik Olofsson

Mattias Walls

Johannes Elg

Piergiorgio Sartor

Sebastian Farås

Roderick Köhle

Roderick Köehle

Bo-Erik Månsson

Zoltan Facius

Paul Springer

Pal Szasz

Daniel Linåker

Thimo Emmerich

Henrik Vallgren

Sebastian Haner

Dennis Harres

Henrik Tuszynski

Henrick Tuszynski

Oliver Vogel

Roderick Koehle

A method of performing sensor fusion of data obtained from a camera () and a supplemental sensor (), the method comprising performing patch tracking () on image data () provided by the camera () to determine tracked patches (), and performing a fusion () of the image data () obtained from the camera () with supplemental data () provided by the supplemental sensor () based on the tracked patches ().

Camera-radar fusion using correspondences

Methods, systems, and computer program products for improving the generation of a 3D representation of an object may include adding a distribution of additional points to plurality of points representative of captured portions of the object based on a computational analysis of the plurality of points, and performing a surface reconstruction on the plurality of points and the distribution of additional points to generate a three-dimensional mesh representation of the object that is watertight.

Methods, devices, and computer program products for improved mesh generation in constructed 3D images

A method of generating a 3D model may include receiving a plurality of 2D images of a physical object captured from a respective plurality of viewpoints in a 3D scan of the physical object in a first process. The method may include receiving a first process 3D mesh representation of the physical object and calculating respective second process estimated position and/or orientation information for each one of the respective plurality of viewpoints of the plurality of 2D images. The method may include generating a second process 3D mesh representation of the physical object using the plurality of 2D images, the second process estimated position and/or orientation information, and the first process 3D mesh representation of the physical object. The method may include generating a 3D model of the physical object by applying surface texture information from the plurality of 2D images to the second process 3D mesh representation of the physical object.

Methods, devices and computer program products for generating 3D models

A method of guiding a user when performing a three-dimensional scan of an object includes determining two-dimensional feature points from two-dimensional captured images of the object, determining three-dimensional coordinates for the two-dimensional feature points, determining a cut plane based on the three-dimensional coordinates that divides the object into a target portion and a cut-away portion, and displaying the target portion of the object on a display using a first indicium and the cut-away portion of the object on the display using a second indicium that is different from the first indicium.

Methods for guiding a user when performing a three dimensional scan and related mobile devices and computer program products

A method for estimating a 3D map is described. The method includes performing an initialization of a plurality of 3D points in the 3D map based on the plurality of 2D images, reprojecting ones of the 3D points onto ones of the 2D images to obtain reprojection points, determining respective reprojection errors of ones of the reprojection points, determining a robust camera parameter associated with a weighting function based on the respective reprojection errors, and performing bundle adjustment to update the 3D map and update the plurality of poses, based on the robust camera parameter and the weighting function. Related systems, devices and computer program products are also described.

Methods, devices and computer program products for global bundle adjustment of 3D images

Methods of determining whether an environment is suitable or acceptable for performing a three-dimensional (3D) scan are provided. The methods may include performing one or more checks on captured image data of the environment, wherein performing each of the one or more checks comprises determining whether the environment satisfies a respective criterion. The method may further include determining that the environment is suitable or unsuitable for performing the 3D scan based on a result of each performed check. Determining that the environment is suitable for performing the 3D scan may include determining that the environment satisfies each of the respective criteria of the performed one or more checks. Determining that the environment is unsuitable for performing the 3D scan may include determining that the environment does not satisfy at least one criterion of the performed one or more checks. Related devices and computer program products are also provided.

Methods, devices, and computer program products for checking environment acceptability for 3D scanning

Image processing devices and methods according to embodiments of the invention advantageously utilize computationally efficient and high speed operations based on dynamic camera models to support rapid determination of multiple camera intrinsics. These built-in dynamic models can utilize varying focus distance values as inputs (i.e., operands) to generate respective intrinsic parameters as outputs, as a stream of images are captured and processed in real time to possibly determine pose and position of a three-dimensional (3D) object during 3 image scanning.

Image processing devices that utilize built-in dynamic camera models to support rapid determination of camera intrinsics and methods of operating same

A method for estimating a 3D map and a plurality of poses is described. The method includes non-recursively performing an initialization of the 3D map and the respective poses of the plurality of poses based on a plurality of first 2D images of the series of 2D images, and recursively performing sequential updating of the 3D map based on recursively updating the 3D map for respective ones of a plurality of second 2D images of the series of 2D images. Related systems, devices and computer program products are also described.

Methods, devices and computer program products for 3D mapping and pose estimation of 3D images

A method of generating a refined depth map for a three-dimensional (3D) model of an object is described. The method may include may include warping a set of target images, resulting in warped target images, wherein the set of target images is selected from the plurality of images; determining a difference between each warped target image and a reference image, wherein the reference image is from the plurality of images; determining, for each warped target image, an alpha mask comprising per-pixel weights; and updating an initialized depth map based on alpha mask, resulting in the estimated depth map. Related systems, devices and computer program products are also described.

Methods, devices and computer program products for gradient based depth reconstructions with robust statistics

An electronic system includes a circuitry configured to obtain a sequence of frames of an object under different viewing angles at consecutive time instances. For a first time instance, the circuitry generates a point cloud descriptive for an external surface of the object on basis of (i) a point cloud obtained for a second time instance preceding the first time instance and (ii) disparity information concerning a frame captured at the first time instance.

Electronic system including image processing unit for reconstructing 3D surfaces and iterative triangulation method

An imaging system includes an optical unit that captures, from a scene, sets of first images as well as sets of second images. The scene is illuminated with non-structured light when the sets of first images are captured and illuminated with structured light when the sets of second images are captured. A pattern processing unit generates, from the second images, a recovered shadow pattern from a shadow pattern projected onto the scene during illumination with structured light, wherein at least one motion-compensated reference image is used. A depth processing unit obtains depth information on the basis of the recovered shadow pattern.

Imaging system using structured light for depth recovery

A method for refining a disparity/motion estimation map of an image at image regions a user has marked, including filtering the image along a mark set by the user, binarizing the filtered image to generate a binarized map of the filtered image, determining a binary step within the binarized map indicating a boundary of an object, and refining the disparity/motion estimation map on the basis of the determined binary step.

Growing community of inventors

Stuttgart, Germany

Francesco Michielin