X Development LLC

Umashankar Nagarajan

Bianca Homberg

Peter Pastor Sampedro

Craig Latimer

Devesh Yamparala

Grasping of an object, by an end effector of a robot, based on a grasp strategy that is selected using one or more machine learning models. The grasp strategy utilized for a given grasp is one of a plurality of candidate grasp strategies. Each candidate grasp strategy defines a different group of one or more values that influence performance of a grasp attempt in a manner that is unique relative to the other grasp strategies. For example, value(s) of a grasp strategy can define a grasp direction for grasping the object (e.g., 'top', 'side'), a grasp type for grasping the object (e.g., &#x201c;pinch&#x201d;, &#x201c;power&#x201d;), grasp force applied in grasping the object, pre-grasp manipulations to be performed on the object, and/or post-grasp manipulations to be performed on the object.

Grasping of an object by a robot based on grasp strategy determined using machine learning model(s)

Methods, systems, and apparatus, including computer-readable media, for robot grasp learning. In some implementations, grasp data describing grasp attempts by robots is received. A set of the grasp attempts that represent unsuccessful grasp attempts is identified. Based on the set of grasp attempts representing unsuccessful grasp attempts, a grasp model based on sensor data for the unsuccessful grasp attempts. After training the grasp model, a performance level of the trained grasp model is verified based on one or more simulations of grasp attempts. In response to verifying the performance level of the trained grasp model, the trained grasp model is provided to one or more robots.

Robot grasp learning

Techniques described herein relate to using reduced-dimensionality embeddings generated from robot sensor data to identify predetermined semantic labels that guide robot interaction with objects. In various implementations, obtaining, from one or more sensors of a robot, sensor data that includes data indicative of an object observed in an environment in which the robot operates. The sensor data may be processed utilizing a first trained machine learning model to generate a first embedded feature vector that maps the data indicative of the object to an embedding space. Nearest neighbor(s) of the first embedded feature vector may be identified in the embedding space. Semantic label(s) may be identified based on the nearest neighbor(s). A given grasp option may be selected from enumerated grasp options previously associated with the semantic label(s). The robot may be operated to interact with the object based on the pose and using the given grasp option.

Robot interaction with objects based on semantic information associated with embedding spaces

Techniques described herein relate to using reduced-dimensionality embeddings generated from robot sensor data to identify predetermined semantic labels that guide robot interaction with objects. In various implementations, sensor data obtained from one or more sensors of a robot includes data indicative of an object observed in an environment in which the robot operates. The sensor data is processed utilizing a first trained machine learning model to generate a first embedded feature vector that maps the data indicative of the object to an embedding space. Nearest neighbor(s) of the first embedded feature vector is identified in the embedding space. Semantic label(s) are identified based on the nearest neighbor(s). A given grasp option is selected from enumerated grasp options previously associated with the semantic label(s). The robot is operated to interact with the object based on the pose and using the given grasp option.

Methods and apparatus related to robot collision avoidance. One method may include: receiving robot instructions to be performed by a robot; at each of a plurality of control cycles of processor(s) of the robot: receiving trajectories to be implemented by actuators of the robot, wherein the trajectories define motion states for the actuators of the robot during the control cycle or a next control cycle, and wherein the trajectories are generated based on the robot instructions; determining, based on a current motion state of the actuators and the trajectories to be implemented, whether implementation of the trajectories by the actuators prevents any collision avoidance trajectory from being achieved; and selectively providing the trajectories or collision avoidance trajectories for operating the actuators of the robot during the control cycle or the next control cycle depending on a result of the determining.

Real time robot collision avoidance

Methods, apparatus, systems, and computer-readable media are provided for detecting a geometric change in a robot's configuration and taking responsive action in instances where the geometric change is likely to impact operation of the robot. In various implementations, a geometric model of a robot in a selected pose may be obtained. Image data of the actual robot in the selected pose may also be obtained. The image data may be compared to the geometric model to detect a geometric difference between the geometric model and the actual robot. Output may be provided that is indicative of the geometric difference between the geometric model and the actual robot.

Detecting and responding to geometric changes to robots

Methods, apparatus, systems, and computer readable media are provided for generating phase synchronized trajectories for actuators of a robot to enable the actuators of the robot to transition from a current motion state to a target motion state. Phase synchronized trajectories produce motion of a reference point of the robot in a one-dimensional straight line in a multi-dimensional space. For example, phase synchronized trajectories of a plurality of actuators that control the movement of an end effector may cause a reference point of the end effector to move in a straight line in Cartesian space. In some implementations, phase synchronized trajectories may be generated and utilized even when those phase synchronized trajectories are less time-optimal than one or more other non-phase synchronized trajectories.

Real time generation of phase synchronized trajectories

Methods, apparatus, systems, and computer readable media are provided for real-time generation of trajectories for actuators of a robot, where the trajectories are generated to lessen the chance of collision with one or more objects in the environment of the robot. In some implementations, a real-time trajectory generator is used to generate trajectories for actuators of a robot based on a current motion state of the actuators, a target motion state of the actuators, and kinematic motion constraints of the actuators. The acceleration constraints and/or other kinematic constraints that are used by the real-time trajectory generator to generate trajectories at a given time are determined so as to lessen the chance of collision with one or more obstacles in the environment of the robot.

Real-time trajectory generation for actuators of a robot to reduce chance of collision with obstacle(s)

Methods, apparatus, systems, and computer readable media are provided for generating updated robot actuator trajectories in response to violation of torque constraints and/or other constraints in previously generated robot actuator trajectories. A real-time trajectory generator is used to generate trajectories for actuators of a robot based on a current motion state of the actuators, a target motion state of the actuators, and kinematic motion constraints of the actuators. The generated trajectory of each of the actuators is analyzed to determine whether a violation of at least one additional constraint occurs. In response to determining violation(s) of the additional constraint, one or more new kinematic motion constraints of the actuators are determined based on the violation(s). The real-time trajectory generator generates updated trajectories based on applying the new kinematic motion constraints in lieu of their counterparts used in generating the trajectories that included the violation(s) of the additional constraint.

Growing community of inventors

Santa Clara, CA, United States of America

Umashankar Nagarajan