Toyota Research Institute, Inc.

Toyota Jidosha Kabushiki Kaisha

Kevin Stone

Krishna Shankar

Jeremy C Ma

Max Bajracharya

Michael Laskey

Brandon Northcutt

Konstantine Mushegian

Katarina Bouma

Umashankar Nagarajan

Josh Petersen

Daniel Helmick

James Borders

Matthew W Amacker

Arshan Poursohi

Samuel Zapolsky

Thomas Kollar

Mark Edward Tjersland

Akiyoshi Ochiai

Kazuto Murase

A method for training an object detection system includes estimating a location of a first object in an environment based on a density cluster map generated from a plurality of images of the environment. The method also includes generating one or more negative training samples of the first object in the environment based on the plurality of images, each of the one or more negative training samples corresponding to a second object at a location in the environment that is different than the estimated location of the first object. The method further includes generating positive training samples from a set of images of the first object. The method also includes training the object detection system to detect the first object based on the positive training samples and the negative training sample.

Object detection improvement based on autonomously selected training samples

A method for 3D object perception is described. The method includes extracting features from each image of a synthetic stereo pair of images. The method also includes generating a low-resolution disparity image based on the features extracted from each image of the synthetic stereo pair images. The method further includes predicting, by a trained neural network, a feature map based on the low-resolution disparity image and one of the synthetic stereo pair of images. The method also includes generating, by a perception prediction head, a perception prediction of a detected 3D object based on the feature map predicted by the trained neural network.

System and method for 3D object perception trained from pure synthetic stereo data

A method for performing a task by a robotic device includes mapping a group of task image pixel descriptors associated with a first group of pixels in a task image of a task environment to a group of teaching image pixel descriptors associated with a second group of pixels in a teaching image based on positioning the robotic device within the task environment. The method also includes determining a relative transform between the task image and the teaching image based on mapping the plurality of task image pixel descriptors. The relative transform indicates a change in one or more of points of 3D space between the task image and the teaching image. The method also includes performing the task associated with the set of parameterized behaviors based on updating one or more parameters of a set of parameterized behaviors associated with the teaching image based on determining the relative transform.

Virtual teach and repeat mobile manipulation system

A method for controlling a robotic device is presented. The method includes capturing an image corresponding to a current view of the robotic device. The method also includes identifying a keyframe image comprising a first set of pixels matching a second set of pixels of the image. The method further includes performing, by the robotic device, a task corresponding to the keyframe image.

Autonomous task performance based on visual embeddings

A method for generating positive and negative training samples is presented. The method includes identifying false positive images of an object based on multiple images of an environment. The method also includes generating positive training samples from a set of images of the object. The method further includes generating a negative training sample from the false positive image. The method still further includes training an object detection system based on the positive training samples and the negative training sample.

A system includes a memory module configured to store image data captured by a camera and an electronic controller communicatively coupled to the memory module. The electronic controller is configured to receive image data captured by the camera, implement a neural network trained to predict a drivable portion in the image data of an environment. The neural network predicts the drivable portion in the image data of the environment. The electronic controller is configured to implement a support vector machine. The support vector machine determines whether the predicted drivable portion of the environment output by the neural network is classified as drivable based on a hyperplane of the support vector machine and output an indication of the drivable portion of the environment.

Embeddings + SVM for teaching traversability

A method for controlling a robotic device is presented. The method includes positioning the robotic device within a task environment. The method also includes mapping descriptors of a task image of a scene in the task environment to a teaching image of a teaching environment. The method further includes defining a relative transform between the task image and the teaching image based on the mapping. Furthermore, the method includes updating parameters of a set of parameterized behaviors based on the relative transform to perform a task corresponding to the teaching image.

A method for training a deep neural network of a robotic device is described. The method includes constructing a 3D model using images captured via a 3D camera of the robotic device in a training environment. The method also includes generating pairs of 3D images from the 3D model by artificially adjusting parameters of the training environment to form manipulated images using the deep neural network. The method further includes processing the pairs of 3D images to form a reference image including embedded descriptors of common objects between the pairs of 3D images. The method also includes using the reference image from training of the neural network to determine correlations to identify detected objects in future images.

Training methods for deep networks

Systems and methods for perception-based tactile sensing in a robot are provided. The robot may include an external portion and an illuminator that outputs illumination. The robot may further include a receiving sensor that receives illumination. The robot may also include a pair of conduits, located at an external portion, that include an injecting conduit that traverses one or more housings of the robot. The injecting conduit may be configured to receive the illumination from the illuminator and output the illumination to illuminate an object external to the robot. The pair of conduits may also include a receiving conduit, traversing one or more housings of the robot, configured to receive the illumination from the object external to the robot and output the illumination to the receiving sensor.

Robots with perception-based fiber-optic tactile sensing and methods for providing the same

Systems, robots, and methods for generating three-dimensional skeleton representations of people are disclosed. A method includes generating, from a two-dimensional image, a two-dimensional skeleton representation of a person present in the two-dimensional image. The two-dimensional skeleton representation includes a plurality of joints and a plurality of links between individual joints of the plurality of joints. The method further includes positioning a cone around one or more links of the plurality of links, and identifying points of a depth cloud that intersect with the cone positioned around the one or more links of the two-dimensional skeleton. The points of the depth cloud are generated by a depth sensor and each point provides depth information. The method also includes projecting the two-dimensional skeleton representation into three-dimensional space using the depth information of the points that intersect with the cone, thereby generating the three-dimensional skeleton representation of the person.

Growing community of inventors

Menlo Park, CA, United States of America

Kevin Stone