Google Inc.

James Robert Bruce

Arshan Poursohi

James Joseph Kuffner

Ryan Michael Hickman

Thor Lewis

Anthony Gerald Francis, Jr

Chaitanya Gharpure

Sumit Jain

Damon Kohler

Christian Frueh

Greg Joseph Klein

Won S Chun

James J Kuffner

Ken Conley

Anthony Gerald Francis

Instructions indicative of changing a view of a virtual object may be received by a device. At least a portion of the virtual object may be viewable from a viewpoint that is at a given distance from a surface of the virtual object. The device may cause a change of the view along a rotational path around the virtual object in response to the receipt of the instructions based on the given distance being greater than a threshold distance. The device may cause a change of the view along a translational path indicative of a shape of the surface of the virtual object in response to the receipt of the instructions based on the given distance being less than the threshold distance.

Methods and systems for viewing a three-dimensional (3D) virtual object

An example method includes determining a point cloud representation of surfaces within an environment. The method further includes providing for display of a graphical interface that shows a model of the surfaces within the environment based on the point cloud representation. The method additionally includes receiving input data indicating one or more positions for one or more virtual sensors on the graphical interface corresponding to one or more physical positions within the environment. The method also includes determining one or more occluded regions within the environment, where the one or more occluded regions are predicted to be occluded from view by one or more sensors positioned at the one or more physical positions within the environment. The method also includes providing for display in the graphical interface of a graphical representation of the one or more occluded regions within the model of the surfaces within the environment.

Sensor placement interface

Systems and devices for acquiring imagery and three-dimensional (3D) models of objects are provided. An example device includes a platform configured to enable an object to be positioned thereon, and a plurality of scanners configured to capture geometry and texture information of the object when the object is positioned on the platform. A first scanner is positioned below the platform so as to capture an image of a portion of an underside of the object, a second scanner is positioned above the platform, and a third scanner is positioned above the platform and offset from a position of the second scanner. The scanners are positioned such that each scanner is outside of a field of view of other scanners. Scanners may include a camera, a light source, and a light-dampening element, and the device may include a control module configured to operate the scanners to individually scan the object.

Systems and devices for acquiring imagery and three-dimensional (3D) models of objects

Within examples, object image masking is provided. An example method includes receiving a depth mask of an object, projecting the depth mask of the object onto an image of the object in a background so as to generate a depth image of the object in the background, determining portions of the depth image of the object in the background that are representative of the object and that are representative of the background, based on the portions of the depth image of the object in the background that are representative of the object determining a foreground mask of the object, and using the foreground mask of the object to identify portions of the image representative of the object.

Object image masking using depth cameras or three-dimensional (3D) models

Methods and systems for comparing a 3D model of a target object to a shape-search database are provided. An example method includes using a mobile device to acquire a plurality of images of a target object, determining a 3D model based on the images, transmitting a search query that includes the 3D model, and receiving a search query result. In another example method, a server could receive a search query that includes a 3D model of an object, compare the 3D model to a shape-search database, generate a search query result based on the comparison, and transmit the search query result. The search query result could include one or more of: information regarding the target object, information regarding one or more objects similar to the target object, and a suggestion for acquiring additional images of the target object.

Using a mobile device for coarse shape matching against cloud-based 3D model database

Methods and systems for controlling light arrays to determine properties of an object are described. An example method includes causing illumination of a surface of an object from multiple illumination positions using a programmable array of lights, and receiving images from an image-capture device while the surface of the object is illuminated. For example, the programmable array of lights may be modulated to cause illumination of a portion of the surface of the object from first and second illumination positions, and a first and second image of the surface of the object captured during illumination from the first and second illumination positions respectively may be received. Subsequently, a processor may determine material information for the object based on an amount of specular reflectivity for the surface of the object and reference to a database of known amounts of specular reflectivity for a plurality of materials.

Controlling light arrays to determine properties of an object

Methods and systems for providing a three-dimensional (3D) image viewer in a webpage are provided. According to an example method, a webpage may be provided, and the webpage may include embedded language that identifies a 3D image viewer to be provided within the webpage. Based on the embedded language, a computer having a processor and a memory may request information associated with rendering a 3D object data model in the 3D image viewer. The method may also include providing the 3D image viewer within the webpage, and receiving information associated with rendering the 3D object data model. Additionally, the 3D object data model may be rendered in the 3D image viewer based on the received information. Additional example systems and methods are described herein.

Embeddable three-dimensional (3D) image viewer

Examples disclose a method and system for merging textures. The method may be executable to receive one or more images of an object, identify a texture value for a point in a first image of the one or more images, and determine a metric indicative of a relation between a view reference point vector and a normal vector of a position of a point on the object relative to the image capturing device. Based on the metrics, the method may be executable to determine a weighted average texture value to apply to a corresponding point of a three-dimensional mesh of the object.

Systems and methods for merging textures

System and methods for rendering three-dimensional (3D) object data models based on a comparison of images. A 3D object data model of an object can be characterized by parameters defining rendering features of the 3D object data model. A comparison can be made of a first rendering of the 3D object data model to one or more reference images related to the object and, based on the comparison, the parameters of the 3D object data model can be modified. Following the modification, the 3D object data model can be rendered to generate a second rendering. Based on the second rendered 3D object data model, statistical information can be obtained and based on the statistical information, the parameters of the 3D object data model can be modified again to further adjust the appearance of the second rendering of the 3D object data model.

Modification of a three-dimensional (3D) object data model based on a comparison of images and statistical information

Methods and systems are provided for determining and transmitting applicable lighting information, applicable viewing perspective, and a 3D model for an object in response to a search query. An example method includes receiving, at a server, a search query regarding an object. A 3D model for the object is determined. The 3D model includes three-dimensional shape information about the object. The method also includes determining, based on a plurality of stored images of the object, at least one applicable light field and at least one applicable viewing perspective. A search query result is transmitted from the server. The search query result may include the 3D model, the applicable light field(s), and the applicable viewing perspective(s). A server and a non-transitory computer readable medium are also disclosed that could perform a similar method.

Determining 3D model information from stored images

A cloud computing system is configured to (i) receive image and environmental data from a computing device, (ii) apply a plurality of image processing algorithms to the received image a plurality of times to generate a corresponding plurality of image processing results, where each application of an image processing algorithm to the received image is executed with a different corresponding parameter set, and (iii) based on the image processing results, select an image processing algorithm and corresponding parameter set for the computing device to use for image processing operations. The cloud computing device may also correlate the results of its analysis with the environmental data received from the computing device, and store the correlation in a machine vision knowledge base for future reference. In some embodiments, the computing device is a component of a robot.

Machine vision calibration with cloud computing systems

Examples disclose a method and system to transform a colored point cloud to a 3D textured mesh. The method may be executable to identify a location on a 2D image of an object, identify a location on a 3D image of the object, and determine a color associated with the location on the 2D image. Determining a color may include receiving data associated with a simulation of a plurality of rays cast on the 3D image, identifying a color of the location on the 3D image associated with the received data, identifying a confidence level associated with the identified color of the location on the 3D image, and associating the identified color of the location on the 3D image with the location on the 2D image.

Growing community of inventors

Sunnyvale, CA, United States of America

James Robert Bruce