Matterport, Inc.

Ebay, Incorporated

Paypal, Inc.

David Alan Gausebeck

Matthew Tschudy Bell

Daniel Ford

Gregory William Coombe

Louis D Marzano

Michael Beebe

Simeon Trieu

Naoto Sakakibara

David Proctor

Kirk Stromberg

Kevin Kane

Simon Wynn

Upendra S Mardikar

Matthew W Mengerink

Vishwanath Shastry

Brian Andrew Phillips

Mykhaylo Kurinnyy

Katherine Woo

Kyle L Simek

William John Brown

Kuo-Yee Richard Lee

Gunnar Hovden

David Lippman

Japjit Tulsi

Babak Robert Shakib

Waleed K Abdulla

Peter Kyuhee Hahn

Abhijit Limaye

This application generally relates to capturing and aligning panoramic image and depth data. In one embodiment, a device is provided that comprises a housing and a plurality of cameras configured to capture two-dimensional images, wherein the cameras are arranged at different positions on the housing and have different azimuth orientations relative to a center point such that the cameras have a collective field-of-view spanning up to 360° horizontally. The device further comprises a plurality of depth detection components configured to capture depth data, wherein the depth detection components are arranged at different positions on the housing and have different azimuth orientations relative to the center point such that the depth detection components have the collective field-of-view spanning up to 360° horizontally.

Capturing and aligning panoramic image and depth data

Systems and techniques for processing and/or transmitting three-dimensional (3D) data are presented. A partitioning component receives captured 3D data associated with a 3D model of an interior environment and partitions the captured 3D data into at least one data chunk associated with at least a first level of detail and a second level of detail. A data component stores 3D data including at least the first level of detail and the second level of detail for the at least one data chunk. An output component transmits a portion of data from the at least one data chunk that is associated with the first level of detail or the second level of detail to a remote client device based on information associated with the first level of detail and the second level of detail.

Processing and/or transmitting 3D data associated with a 3D model of an interior environment

An environmental capture system (ECS) captures image data and depth information in a 360-degree scene. The captured image data and depth information can be used to generate a 360-degree scene. The ECS comprises a frame, a drive train mounted to the frame, and an image capture device coupled to the drive train to capture, while pointed in a first direction, a plurality of images at different exposures in a first field of view (FOV) of the 360-degree scene. The ECS further comprises a depth information capture device coupled to the drive train. The depth information capture device and the image capture device are rotated by the drive train about a first, substantially vertical, axis from the first direction to a second direction. The depth information capture device, while being rotated from the first direction to the second direction, captures depth information for a first portion of the 360-degree scene. The image capture device captures, while pointed in the second direction, a plurality of images at different exposures in a second FOV that overlaps the first FOV of the 360-degree scene. The depth information capture device and the image capture device are rotated by the drive train about the first axis from the second direction to a third direction. The depth information capture device, while being rotated from the second direction to the third direction, captures depth information for a second portion of the 360-degree scene. The image capture device, while pointed in the third direction, captures a plurality of images at different exposures in a third FOV that overlaps the second FOV of the 360-degree scene.

Systems and methods for capturing and generating panoramic three-dimensional models and images

An apparatus comprising a housing, a mount configured to be coupled to a motor to horizontally move the apparatus, a wide-angle lens coupled to the housing, the wide-angle lens being positioned above the mount thereby being along an axis of rotation, the axis of rotation being the axis along which the apparatus rotates, an image capture device within the housing, the image capture device configured to receive two-dimensional images through the wide-angle lens of environment, and a LiDAR device within the housing, the LiDAR device configured to generate depth data based on the environment.

System and method of capturing and generating panoramic three-dimensional images

An example system comprises a base with a bottom end and a top end opposite the bottom end, a motor within the base, the motor being coupled to a rotational component of the base and configured to turn the rotational component about an axis of rotation, the rotational component being at the top end of the base, the axis of rotation being perpendicular to the top end of the base, and an arm coupled to the rotational component, the arm configured to move a holding member above the top of the base, the holding member configurable to hold a digital device above the top end of the base, the arm being adjustable to position and tilt the holding member, the rotational component being capable of turning the arm and the holding member, the arm configured to tilt the holding member at a first angle relative to the arm.

Motor mount for image capture of surrounding environment

The disclosed subject matter is directed to employing machine learning models configured to predict 3D data from 2D images using deep learning techniques to derive 3D data for the 2D images. In some embodiments, a method is provided that comprises receiving, by a system comprising a processor, a panoramic image, and employing, by the system, a three-dimensional data from two-dimensional data (3D-from-2D) convolutional neural network model to derive three-dimensional data from the panoramic image, wherein the 3D-from-2D convolutional neural network model employs convolutional layers that wrap around the panoramic image as projected on a two-dimensional plane to facilitate deriving the three-dimensional data.

Employing three-dimensional (3D) data predicted from two-dimensional (2D) images using neural networks for 3D modeling applications and other applications

Systems and methods for building a three-dimensional composite scene are disclosed. Certain embodiments of the systems and methods may include the use of a three-dimensional capture device that captures a plurality of three-dimensional images of an environment. Some embodiments may further include elements concerning aligning and/or mapping the captured images. Various embodiments may further include elements concerning reconstructing the environment from which the images were captured. The methods disclosed herein may be performed by a program embodied on a non-transitory computer-readable storage medium when executed the program is executed a processor.

Capturing and aligning three-dimensional scenes

This application generally relates to capturing and aligning panoramic image and depth data. In one embodiment, a device is provided that comprises a housing and a plurality of cameras configured to capture two-dimensional images, wherein the cameras are arranged at different positions on the housing and have different azimuth orientations relative to a center point such that the cameras have a collective field-of-view spanning up to 360&#xb0; horizontally. The device further comprises a plurality of depth detection components configured to capture depth data, wherein the depth detection components are arranged at different positions on the housing and have different azimuth orientations relative to the center point such that the depth detection components have the collective field-of-view spanning up to 360&#xb0; horizontally.

Systems and methods for generating three-dimensional models with correlated three-dimensional and two dimensional imagery data are provided. In particular, imagery data can be captured in two dimensions and three dimensions. Imagery data can be transformed into models. Two-dimensional data and three-dimensional data can be correlated within models. Two-dimensional data can be selected for display within a three-dimensional model. Modifications can be made to the three-dimensional model and can be displayed within a three-dimensional model or within two-dimensional data. Models can transition between two dimensional imagery data and three dimensional imagery data.

Selecting two-dimensional imagery data for display within a three-dimensional model

Growing community of inventors

Mountain View, CA, United States of America

David Alan Gausebeck