Sony Group Corporation

Sony Corporation

Sony Mobile Communications Inc.

Johannes Elg

Fredrik Mattisson

Fredrik Olofsson

Lars Novak

Daniel Linåker

Mattias Walls

Jonas Gustavsson

Pal Szasz

Francesco Michielin

Mats Wernersson

Sebastian Farås

Henrik Vallgren

Bo-Erik Månsson

Ola Karl Thörn

Sebastian Haner

Anders Wikander

Roderick Köhle

Allan Johansson

Henrik Tuszynski

Henrick Tuszynski

Ola Thörn

An electronic device includes a processor and a memory coupled to the processor. The memory includes a computer readable storage medium having computer readable program instructions stored therein that are executable by the processor to determine a position and/or orientation of sensor device that is arranged to capture data for a subject from the position and/or at the orientation, and, based on the position and/or orientation that was determined, control creation of a 3D representation of the subject from the data captured by the sensor device. Related methods and computer program products are also discussed.

Electronic devices, methods, and computer program products for controlling 3D modeling operations based on pose metrics

Methods of creating a texture for a three-dimensional (3D) model using frequency separation and/or depth buffers are provided. Frequency separation may include splitting one or more images each into higher frequency components (which includes finer details such-as facial pores, lines, birthmarks, spots, or other textural details) and lower frequency components (such as color or tone). Depth buffering may include storing higher frequency components of the images within a depth buffer based on a distance of a corresponding vertex in the 3D model from the camera coordinate system, and then, using the closest pixel to the camera. This pixel likely has a highest amount of sharpness or detail. The lower frequency components can be averaged to account for illumination differences, but because the high frequency components have been separated, detail in the final texture may be preserved. Related devices and computer program products are also provided.

Methods, devices, and computer program products for 3D texturing

A method of guiding a user when performing a three-dimensional scan of an object includes determining two-dimensional feature points from two-dimensional captured images of the object, determining three-dimensional coordinates for the two-dimensional feature points, determining a cut plane based on the three-dimensional coordinates that divides the object into a target portion and a cut-away portion, and displaying the target portion of the object on a display using a first indicium and the cut-away portion of the object on the display using a second indicium that is different from the first indicium.

Methods for guiding a user when performing a three dimensional scan and related mobile devices and computer program products

Methods of constructing a printable three-dimensional (3D) model are provided. A method of constructing a printable 3D model includes constructing a ray-cast mesh representation of an object by using ray-casting on a preliminary digital 3D model of the object. Moreover, the method includes converting the preliminary digital 3D model of the object into the printable 3D model, of the object by adding, to the preliminary digital 3D model of the object, an interior surface that is based on the ray-cast mesh representation. Related devices and computer program products are also provided.

Methods of constructing a printable 3D model, and related devices and computer program products

A method for creating a 3D image is described. The method includes receiving a first 2D image and a second 2D image, identifying a plurality of first feature points in the first 2D image and a corresponding plurality of second feature points in the second 2D image, estimating a plurality of distances based on corresponding ones of the plurality of first feature points and based on corresponding ones of the plurality of second feature points, determining a mean and a standard deviation of inverses of the plurality of distances that were estimated, generating a baseline initialization coordinates based on the mean and the standard deviation of inverses of the plurality of distances, generating the 3D image based on the baseline initialization coordinates, and providing the 3D image for display. Related systems, devices and computer program products are also described.

Methods, devices and computer program products using feature points for generating 3D images

Methods of determining whether an environment is suitable or acceptable for performing a three-dimensional (3D) scan are provided. The methods may include performing one or more checks on captured image data of the environment, wherein performing each of the one or more checks comprises determining whether the environment satisfies a respective criterion. The method may further include determining that the environment is suitable or unsuitable for performing the 3D scan based on a result of each performed check. Determining that the environment is suitable for performing the 3D scan may include determining that the environment satisfies each of the respective criteria of the performed one or more checks. Determining that the environment is unsuitable for performing the 3D scan may include determining that the environment does not satisfy at least one criterion of the performed one or more checks. Related devices and computer program products are also provided.

Methods, devices, and computer program products for checking environment acceptability for 3D scanning

Image processing devices utilize computationally efficient operations based on exposure time and object speed to detect and estimate motion blur. These operations include determining, from a plurality of image frames of an object, a representative length of movement associated with at least one feature point of the object, which undergoes movement between the plurality of image frames. This representative length of movement is converted to an estimate of motion blur for a respective image frame, using operations that are function of length of movement and exposure time.

Image processing devices with efficient motion blur detection and methods of operating same

A method for determining the scale of a digital 3D model may include receiving, corresponding to a plurality of images, a respective plurality of first distances between the camera and the object in the environment and calculating, using a simultaneous localization and mapping (SLAM) algorithm, a plurality of second distances between the camera and the object in the 3D model of the environment. The method may include calculating a plurality of ratios corresponding to the plurality of images based on respective ones of the plurality of first distances and respective ones of the second distances and determining a scale of the 3D model based on the plurality of ratios. The method may include creating a scaled digital 3D model based on the 3 D model and the determined scale of the 3 D model. Distances and sizes in the scaled 3D model may correspond to actual distances and sizes of the environment.

Simultaneous localization and mapping (SLAM) devices with scale determination and methods of operating the same

A method for estimating a 3D map and a plurality of poses is described. The method includes non-recursively performing an initialization of the 3D map and the respective poses of the plurality of poses based on a plurality of first 2D images of the series of 2D images, and recursively performing sequential updating of the 3D map based on recursively updating the 3D map for respective ones of a plurality of second 2D images of the series of 2D images. Related systems, devices and computer program products are also described.

Methods, devices and computer program products for 3D mapping and pose estimation of 3D images

Methods, systems and computer program products for the generation of 3D images may include receiving a plurality of points that correspond to a surface of an object, generating a first mesh representation of a shape of the object, the first mesh representation comprising a plurality of polygons, respective ones of the plurality of polygons comprising at least three vertices and at least three edges, associating the plurality of points with a first set of the plurality of polygons, classifying a portion of the plurality of points as background points, and adjusting the first mesh representation of the object to create a second mesh representation of the object by removing from the first mesh representation at least one of the first set of the plurality of polygons that is associated with the portion of the plurality of points that are classified as the background points.

Methods, devices and computer program products for generation of mesh in constructed 3D images

A method for creating a 3D image is described. The method includes receiving a plurality of 2D images of an object, identifying respective 2D positions of one or more features in each of the plurality of 2D images, generating a 3D representation of the object, based on the respective 2D positions of the one or more features in each of the plurality of 2D images, and determining a 3D angular orientation of the 3D representation of the object based on relative positions of the one or more features in the 3D representation of the object. Related systems, devices and computer program products are also described.

Methods, devices and computer program products for creating textured 3D images

Methods, systems, and computer program products for removing unused portions of a 3D mesh representation of an object may include generating a first mesh representation of the object, the first mesh representation including a plurality of polygons, respective ones of the polygons including at least three vertices and at least three edges, wherein respective ones of the plurality of polygons are associated with a precision value that indicates an extent to which the respective ones of the plurality of polygons in the first mesh representation match the object, and adjusting the first mesh representation of the object to create a second mesh representation of the object by removing, front the first mesh representation, polygons of the plurality of polygons that are associated with precision values that have not been modified from an initial precision value.

Methods, devices and computer program products for generation of mesh in constructed 3D images containing incomplete information

The invention is directed to exposure control in a camera. An exemplary method comprises determining depth information associated with a portion of an image frame; obtaining exposure data associated with the portion of the image frame; and controlling an amount of exposure for the portion of the image frame based on the depth information and the exposure data.

Exposure control using depth information

The invention is directed to systems, methods and computer program products for capturing an image using an array camera. A method comprises determining an application associated with capturing an image using an array camera, wherein the array camera comprises a first sensor and at least one second sensor, wherein the first sensor comprises a red filter, a green filter, and a blue filter, and wherein each second sensor comprises a red filter, a green filter, or a blue filter; determining whether the application requires the image to have a first resolution equal to or greater than a predetermined resolution; determining whether the application requires depth information associated with the image; and in response to determining the application does not require the image to have the first resolution and does not require depth information, activating the first sensor, and capturing the image using the first sensor.

Array camera design with dedicated Bayer camera

The invention is directed to determining compensation distance for capturing an image in a zoom mode using a camera. An exemplary method comprises determining the camera is in a non-zoom mode; receiving first depth information associated with an object in the image when the camera is in the non-zoom mode; receiving first angle information associated with the object in the image when the camera is in the non-zoom mode; switching from the non-zoom mode to a zoom mode; receiving second angle information associated with the object in the image when the camera is in the zoom mode; determining a compensation distance for capturing the image of the object in the zoom mode based on the first depth information, the first angle information, and the second angle information; and generate an image based on the compensated distance.

Growing community of inventors

Helsingborg, Sweden

Johannes Elg