Matterport, Inc.

Intellectual Ventures Holding 67 LLC

Reactrix Systems, Inc.

Other

Facebook, Inc.

Intellectual Ventures Holding 81 LLC

Microsoft Technology Licensing, LLC

Intellectual Ventures Holdings 81 LLC

Matthew Tschudy Bell

David Alan Gausebeck

Daniel Ford

Gregory William Coombe

Michael Beebe

Mykhaylo Kurinnyy

Tipatat Chennavasin

Charles H Clanton

Michael Hulme

Eyal Ophir

Matthew Vieta

Philip Gleckman

Daniel Prochazka

James Mildrew

Helen Shaughnessy

Preston Cowley

Lester Lee

Peter McColgan

Brian Schulman

Dustin Michael Cook

James Sundra

Alan Tan

Joshua Zide

Gunnar Hovden

Kevin Allen Bjorke

Kyle L Simek

William John Brown

Dave Gausebeck

Russell H Belfer

Steve Fink

Babak Robert Shakib

Malik Coates

Scott Anthony Bylo Chacon

Peter Kyuhee Hahn

Nick Hengeveld

Haakon Erichsen

Waleed K Abdulla

David V Buchhofer, Jr

This application generally relates to capturing and aligning panoramic image and depth data. In one embodiment, a device is provided that comprises a housing and a plurality of cameras configured to capture two-dimensional images, wherein the cameras are arranged at different positions on the housing and have different azimuth orientations relative to a center point such that the cameras have a collective field-of-view spanning up to 360° horizontally. The device further comprises a plurality of depth detection components configured to capture depth data, wherein the depth detection components are arranged at different positions on the housing and have different azimuth orientations relative to the center point such that the depth detection components have the collective field-of-view spanning up to 360° horizontally.

Capturing and aligning panoramic image and depth data

Systems and techniques for processing and/or transmitting three-dimensional (3D) data are presented. A partitioning component receives captured 3D data associated with a 3D model of an interior environment and partitions the captured 3D data into at least one data chunk associated with at least a first level of detail and a second level of detail. A data component stores 3D data including at least the first level of detail and the second level of detail for the at least one data chunk. An output component transmits a portion of data from the at least one data chunk that is associated with the first level of detail or the second level of detail to a remote client device based on information associated with the first level of detail and the second level of detail.

Processing and/or transmitting 3D data associated with a 3D model of an interior environment

This application generally relates to defining, displaying and interacting with tags in a 3D model. In an embodiment, a method includes generating, by a system including a processor, a three-dimensional model of an environment based on sets of aligned three-dimensional data captured from the environment, and associating tags with defined locations of the three-dimensional model, wherein the tags are respectively represented by tag icons that are spatially aligned with the defined locations of the three-dimensional model as included in different representations of the three-dimensional model rendered via an interface of a device, wherein the different representations correspond to different perspectives of the three-dimensional model, and wherein selection of the tag icons causes the tags respectively associated therewith to be rendered at the device.

Defining, displaying and interacting with tags in a three-dimensional model

Systems and methods for building a three-dimensional composite scene are disclosed. Certain embodiments of the systems and methods may include the use of a three-dimensional capture device that captures a plurality of three-dimensional images of an environment. Some embodiments may further include elements concerning aligning and/or mapping the captured images. Various embodiments may further include elements concerning reconstructing the environment from which the images were captured. The methods disclosed herein may be performed by a program embodied on a non-transitory computer-readable storage medium when executed the program is executed a processor.

Capturing and aligning three-dimensional scenes

This application generally relates to capturing and aligning panoramic image and depth data. In one embodiment, a device is provided that comprises a housing and a plurality of cameras configured to capture two-dimensional images, wherein the cameras are arranged at different positions on the housing and have different azimuth orientations relative to a center point such that the cameras have a collective field-of-view spanning up to 360&#xb0; horizontally. The device further comprises a plurality of depth detection components configured to capture depth data, wherein the depth detection components are arranged at different positions on the housing and have different azimuth orientations relative to the center point such that the depth detection components have the collective field-of-view spanning up to 360&#xb0; horizontally.

Systems and methods for generating three-dimensional models with correlated three-dimensional and two dimensional imagery data are provided. In particular, imagery data can be captured in two dimensions and three dimensions. Imagery data can be transformed into models. Two-dimensional data and three-dimensional data can be correlated within models. Two-dimensional data can be selected for display within a three-dimensional model. Modifications can be made to the three-dimensional model and can be displayed within a three-dimensional model or within two-dimensional data. Models can transition between two dimensional imagery data and three dimensional imagery data.

Selecting two-dimensional imagery data for display within a three-dimensional model

The present disclosure concerns a methodology that allows a user to 'orbit' around a model on a specific axis of rotation and view an orthographic floor plan of the model. A user may view and 'walk through' the model while staying at a specific height above the ground with smooth transitions between orbiting, floor plan, and walking modes.

Multi-modal method for interacting with 3D models

The disclosed subject matter is directed to employing machine learning models configured to predict 3D data from 2D images using deep learning techniques to derive 3D data for the 2D images. In some embodiments, a system is described comprising a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory. The computer executable components comprise a reception component configured to receive two-dimensional images, and a three-dimensional data derivation component configured to employ one or more three-dimensional data from two-dimensional data (3D-from-2D) neural network models to derive three-dimensional data for the two-dimensional images.

Employing three-dimensional (3D) data predicted from two-dimensional (2D) images using neural networks for 3D modeling applications and other applications

Systems and methods for generating three-dimensional models having regions of various resolutions are provided. In particular, imagery data can be captured and utilized to generate three-dimensional models. Regions of texture can be mapped to regions of a three-dimensional model when rendered. Resolutions of texture can be selectively altered and regions of texture can be selectively segmented to reduce texture memory cost. Texture can be algorithmically generated based on alternative texturing techniques. Models can be rendered having regions at various resolutions.

Optimal texture memory allocation

Systems and methods for processing gesture-based user interactions within an interactive display area are provided. The display of one or more virtual objects and user interactions with the one or more virtual objects may be further provided. Multiple interactive areas may be created by partitioning an area proximate a display into multiple volumetric spaces or zones. The zones may be associated with respective user interaction capabilities. A representation of a user on the display may change as the ability of the user to interact with one or more virtual object changes.

Processing of gesture-based user interaction using volumetric zones

Growing community of inventors

Palo Alto, CA, United States of America

Matthew Tschudy Bell