Google Inc.

X Development LLC

Other

Anthony Gerald Francis, Jr

James Joseph Kuffner

Ryan Michael Hickman

Arshan Poursohi

Thor Lewis

Chaitanya Gharpure

James Robert Bruce

Peter Elving Anderson-Sprecher

Damon Kohler

Andrew Logan Harp

Won S Chun

The present application discloses systems and methods for inventorying objects. In one embodiment, a robot detects an object and sends identification data and location data associated with the detected object to a cloud computing system. The identification data may include an image of the object and/or information from a tag, code, or beacon associated with the object. In response to receiving the identification data and the location data, the cloud computing system identifies the object. The cloud computing system may also determine or create a first map associated with the identified object and a second map associated with the identified object. The first map may be associated with the current location of the object and the second map may correspond to a past location of the object. The cloud computing server may compare the first and second maps, and then send instructions to the robot based on the comparison.

Systems and methods for inventorying objects

Methods and systems for comparing a 3D model of a target object to a shape-search database are provided. An example method includes using a mobile device to acquire a plurality of images of a target object, determining a 3D model based on the images, transmitting a search query that includes the 3D model, and receiving a search query result. In another example method, a server could receive a search query that includes a 3D model of an object, compare the 3D model to a shape-search database, generate a search query result based on the comparison, and transmit the search query result. The search query result could include one or more of: information regarding the target object, information regarding one or more objects similar to the target object, and a suggestion for acquiring additional images of the target object.

Using a mobile device for coarse shape matching against cloud-based 3D model database

Methods and systems for determining the shape of an object based on shadows cast by the object are described. An example method may include receiving a plurality of images of an object casting a shadow. Each image may include a shadow cast by the object as the object is illuminated by a light source that moves over a plurality of positions. The method may further include determining, by a computing device, respectively for each image of the plurality of images a two-dimensional (2D) silhouette of the object and a respective position of the light source relative to the object. According to the method, a three-dimensional (3D) object data model of the object may be generated by the computing device based on the 2D silhouette of the object and the respective position of the light source relative to the object for each image of the plurality of images.

Systems and methods for capturing aspects of objects using images and shadowing

A server may receive a search query indicative of an object. The server may identify a 3D model from within a database of 3D object data models that corresponds to the object. The 3D model may be associated with information that pertains to the object. The server may determine an updated search query based on the information associated with the 3D model. The updated search query may be indicative of one or more of an environment of the object, an orientation of the object in the environment, a position of the object in the environment, or an identification of one or more other objects in the environment. The server may provide one or more search query results that include data pertaining to the object based on the updated search query.

Methods and systems for use of a database of three-dimensional (3D) object data models for search queries

Methods and systems for robot cloud computing are described. Within examples, cloud-based computing generally refers to networked computer architectures in which application execution and storage may be divided, to some extent, between client and server devices. A robot may be any device that has a computing ability and interacts with its surroundings with an actuation capability (e.g., electromechanical capabilities). A client device may be configured as a robot including various sensors and devices in the forms of modules, and different modules may be added or removed from robot depending on requirements. In some example, a robot may be configured to receive a second device, such as mobile phone, that may be configured to function as an accessory or a 'brain' of the robot. A robot may interact with the cloud to perform any number of actions, such as to share information with other cloud computing devices.

Projection of interactive map data

A method includes receiving first sensor data acquired by a first sensor in communication with a cloud computing system. The first sensor data has a first set of associated attributes including a time and a location at which the first sensor data was acquired. The method also includes receiving second sensor data acquired by a second sensor in communication with the cloud computing system. The second data has a second set of associated attributes including a time and a location at which the second sensor data was acquire. Further, the method includes generating a data processing result based at least in part on the first sensor data, the first set of associated attributes, the second sensor data, and the second set of associated attributes and instructing a robot in communication with the cloud computing system to perform a task based at least in part on the data processing result.

Enhancing sensor data by coordinating and/or correlating data attributes

An example method may involve a computing device receiving low-resolution images of an object and high-resolution images of the object. The method may also involve causing a 3D image viewer to display a first high-resolution image of the object. The method may further involve receiving an input associated with adjusting a view of the object in the 3D image viewer. The method may still further involve, based on the input, causing the 3D image viewer to sequentially display at least a portion of the low-resolution images so as to adjust the view of the object in the 3D image viewer. The method may yet still further involve, in response to a completion of the sequential display, causing the 3D image viewer to display a second high-resolution image of the object that corresponds to a respective low-resolution image of the object displayed at the completion of the sequential display.

Methods and systems for viewing dynamic high-resolution 3D imagery over a network

Methods and systems for providing a three-dimensional (3D) image viewer in a webpage are provided. According to an example method, a webpage may be provided, and the webpage may include embedded language that identifies a 3D image viewer to be provided within the webpage. Based on the embedded language, a computer having a processor and a memory may request information associated with rendering a 3D object data model in the 3D image viewer. The method may also include providing the 3D image viewer within the webpage, and receiving information associated with rendering the 3D object data model. Additionally, the 3D object data model may be rendered in the 3D image viewer based on the received information. Additional example systems and methods are described herein.

Embeddable three-dimensional (3D) image viewer

Methods and systems for image acquisition are described. In an example, a computing device may be configured to cause incremental relative rotation of a projector with respect to an image-capture device through multiple discrete angles. At each angle of the multiple discrete angles, the computing device may be configured to cause the image-capture device to capture a first image of an object, cause the projector to project a pattern on the object, and cause the image-capture device, while the pattern is projected on the object, to capture a second image of the object.

Systems and methods for image acquisition

Methods and systems for image marking and generation of a three-dimensional (3D) image of an object are described. In an example, a computing device may be configured to receive a first set of images of an object that capture details of the object. The computing device may also be configured to receive a second set of images that include markings projected on the object and that are indexed to correspond to images of the first set of images. The computing device may be configured to spatially align images of the second set of images based on the markings projected on the object and determine respective images of the first set of images corresponding to spatially aligned images of the second set of images. The computing device may then generate a 3D image of the object from the respective images of the first set of images.

Systems and methods for marking images for three-dimensional image generation

System and methods for rendering three-dimensional (3D) object data models based on a comparison of images. A 3D object data model of an object can be characterized by parameters defining rendering features of the 3D object data model. A comparison can be made of a first rendering of the 3D object data model to one or more reference images related to the object and, based on the comparison, the parameters of the 3D object data model can be modified. Following the modification, the 3D object data model can be rendered to generate a second rendering. Based on the second rendered 3D object data model, statistical information can be obtained and based on the statistical information, the parameters of the 3D object data model can be modified again to further adjust the appearance of the second rendering of the 3D object data model.

Modification of a three-dimensional (3D) object data model based on a comparison of images and statistical information

Methods and systems for robot and user interaction are provided to generate a personality for the robot. A robot may access a user device to determine or identify information about a user, and the robot may be configured to tailor a personality for interaction with the user based on the identified information. A robot may further receive data associated with the user to identify the user, such as using speech or face recognition. The robot may provide a personalized interaction or response to the user based on the determined information of the user. In some examples, a robot's personality or personalization can be transferred from one robot to another robot, or information stored on one robot can be shared with another robot over the cloud.

Methods and systems for robot personality development

Examples disclose systems and methods for recognizing objects. A method may be executable to receive a query from a robot. The query may include identification data associated with an object and contextual data associated with the object. The query may also include situational data. The method may also be executable to identify the object based at least in part on the data in the query received from the robot. Further, the method may be executable to send data associated with the identified object to the robot in response to the query.

Systems and methods for object recognition

Methods and systems for interacting with multiple three-dimensional (3D) object data models are provided. An example method may involve providing to a display device for display a first 3D object data model and a second 3D object data model. Information associated with a modification to the first 3D object data model may be received. Based on the received information, a same change may be applied to the first 3D object data model and applied to the second 3D object data model to obtain a first modified 3D object data model and a second modified 3D object data model. According to the method, the first modified 3D object data model and the second modified 3D object data model may be provided to the display device for substantially simultaneous display.

Growing community of inventors

San Jose, CA, United States of America

Anthony Gerald Francis, Jr