Toyota Research Institute, Inc.

California Institute of Technology

Jeremy C Ma

Krishna Shankar

Kevin Stone

Max Bajracharya

Umashankar Nagarajan

Michael Laskey

Josh Petersen

Daniel Helmick

James Borders

Paul W Hebert

Brett A Kennedy

James W Borders

Charles F Bergh

Nicolas H Hudson

A method for performing a task by a robotic device includes mapping a group of task image pixel descriptors associated with a first group of pixels in a task image of a task environment to a group of teaching image pixel descriptors associated with a second group of pixels in a teaching image based on positioning the robotic device within the task environment. The method also includes determining a relative transform between the task image and the teaching image based on mapping the plurality of task image pixel descriptors. The relative transform indicates a change in one or more of points of 3D space between the task image and the teaching image. The method also includes performing the task associated with the set of parameterized behaviors based on updating one or more parameters of a set of parameterized behaviors associated with the teaching image based on determining the relative transform.

Virtual teach and repeat mobile manipulation system

A method for controlling a robotic device is presented. The method includes capturing an image corresponding to a current view of the robotic device. The method also includes identifying a keyframe image comprising a first set of pixels matching a second set of pixels of the image. The method further includes performing, by the robotic device, a task corresponding to the keyframe image.

Autonomous task performance based on visual embeddings

A method for visualizing data generated by a robotic device is presented. The method includes displaying an intended path of the robotic device in an environment. The method also includes displaying a first area in the environment identified as drivable for the robotic device. The method further includes receiving an input to identify a second area in the environment as drivable and transmitting the second area to the robotic device.

System and method for augmenting a visual output from a robotic device

A system includes a memory module configured to store image data captured by a camera and an electronic controller communicatively coupled to the memory module. The electronic controller is configured to receive image data captured by the camera, implement a neural network trained to predict a drivable portion in the image data of an environment. The neural network predicts the drivable portion in the image data of the environment. The electronic controller is configured to implement a support vector machine. The support vector machine determines whether the predicted drivable portion of the environment output by the neural network is classified as drivable based on a hyperplane of the support vector machine and output an indication of the drivable portion of the environment.

Embeddings + SVM for teaching traversability

A method for controlling a robotic device is presented. The method includes positioning the robotic device within a task environment. The method also includes mapping descriptors of a task image of a scene in the task environment to a teaching image of a teaching environment. The method further includes defining a relative transform between the task image and the teaching image based on the mapping. Furthermore, the method includes updating parameters of a set of parameterized behaviors based on the relative transform to perform a task corresponding to the teaching image.

Robotics platforms in accordance with various embodiments of the invention can be utilized to implement highly dexterous robots capable of whole body motion. Robotics platforms in accordance with one embodiment of the invention include: a memory containing a whole body motion application; a spine, where the spine has seven degrees of freedom and comprises a spine actuator and three spine elbow joints that each include two spine joint actuators; at least one limb, where the at least one limb comprises a limb actuator and three limb elbow joints that each include two limb joint actuators; a tracked base; a connecting structure that connects the at least one limb to the spine; a second connecting structure that connects the spine to the tracked base; wherein the processor is configured by the whole body motion application to move the at least one limb and the spine to perform whole body motion.

Growing community of inventors

Pasadena, CA, United States of America

Jeremy C Ma