Toyota Research Institute, Inc.

X Development LLC

Google Inc.

Advanced Micro Devices Corporation

Krishna Shankar

Kevin Stone

Nicolas Hudson

Jeremy C Ma

Max Bajracharya

Alexander Toshev

Michael Laskey

Nareshkumar Rajkumar

Patrick Christopher Leger

Rainer Hessmer

Umashankar Nagarajan

Josh Petersen

Daniel Helmick

James Borders

Ram Ramani

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for sharing learned information among robots. In some implementations, a robot obtains sensor data indicating characteristics of an object. The robot determines a classification for the object and generates an embedding for the object using a machine learning model stored by the robot. The robot stores the generated embedding and data indicating the classification for the object. The robot sends the generated embedding and the data indicating the classification to a server system. The robot receives, from the server system, an embedding generated by a second robot and a corresponding classification. The robot stores the received embedding and the corresponding classification in the local cache of the robot. The robot may then use the information in the cache to identify objects.

Sharing learned information among robots

Methods, apparatus, and computer-readable media for determining and utilizing corrections to robot actions. Some implementations are directed to updating a local features model of a robot in response to determining a human correction of an action performed by the robot. The local features model is used to determine, based on an embedding generated over a corresponding neural network model, one or more features that are most similar to the generated embedding. Updating the local features model in response to a human correction can include updating a feature embedding, of the local features model, that corresponds to the human correction. Adjustment(s) to the features model can immediately improve robot performance without necessitating retraining of the corresponding neural network model.

Update of local features model based on correction to robot action

A method for performing a task by a robotic device includes mapping a group of task image pixel descriptors associated with a first group of pixels in a task image of a task environment to a group of teaching image pixel descriptors associated with a second group of pixels in a teaching image based on positioning the robotic device within the task environment. The method also includes determining a relative transform between the task image and the teaching image based on mapping the plurality of task image pixel descriptors. The relative transform indicates a change in one or more of points of 3D space between the task image and the teaching image. The method also includes performing the task associated with the set of parameterized behaviors based on updating one or more parameters of a set of parameterized behaviors associated with the teaching image based on determining the relative transform.

Virtual teach and repeat mobile manipulation system

A method for controlling a robotic device is presented. The method includes capturing an image corresponding to a current view of the robotic device. The method also includes identifying a keyframe image comprising a first set of pixels matching a second set of pixels of the image. The method further includes performing, by the robotic device, a task corresponding to the keyframe image.

Autonomous task performance based on visual embeddings

A system includes a memory module configured to store image data captured by a camera and an electronic controller communicatively coupled to the memory module. The electronic controller is configured to receive image data captured by the camera, implement a neural network trained to predict a drivable portion in the image data of an environment. The neural network predicts the drivable portion in the image data of the environment. The electronic controller is configured to implement a support vector machine. The support vector machine determines whether the predicted drivable portion of the environment output by the neural network is classified as drivable based on a hyperplane of the support vector machine and output an indication of the drivable portion of the environment.

Embeddings + SVM for teaching traversability

A method for controlling a robotic device is presented. The method includes positioning the robotic device within a task environment. The method also includes mapping descriptors of a task image of a scene in the task environment to a teaching image of a teaching environment. The method further includes defining a relative transform between the task image and the teaching image based on the mapping. Furthermore, the method includes updating parameters of a set of parameterized behaviors based on the relative transform to perform a task corresponding to the teaching image.

A method for identifying objects includes generating a feature vector of an unknown object from an image of an environment. The method also includes comparing the feature vector of the unknown object to feature vectors of known objects. The method further includes determining whether a similarity between the feature vector of the unknown object and the feature vector of one of the known objects satisfies a threshold. Furthermore, the method includes identifying the unknown object based on the determination.

Unknown object identification for robotic device

A method for training a deep neural network of a robotic device is described. The method includes constructing a 3D model using images captured via a 3D camera of the robotic device in a training environment. The method also includes generating pairs of 3D images from the 3D model by artificially adjusting parameters of the training environment to form manipulated images using the deep neural network. The method further includes processing the pairs of 3D images to form a reference image including embedded descriptors of common objects between the pairs of 3D images. The method also includes using the reference image from training of the neural network to determine correlations to identify detected objects in future images.

Training methods for deep networks

Construction of a novel multilayer conductive interconnection for an integrated circuit having more than one conductive layer is disclosed comprising a lower barrier layer which may be in contact with an underlying silicon substrate and comprising a material selected from the class consisting of TiW and TiN; an intermediate layer of conductive metal such as an aluminum base metal; and an upper barrier layer which may be in contact with a second aluminum base metal layer and which is selected from the class consisting of TiW, TiN, MoSi.sub.x and TaSi where x equals 2 or more.

Growing community of inventors

Los Altos, CA, United States of America

Krishna Shankar