Industrial Perception, Inc.

X Development LLC

Intrinsic Innovation LLC

Hauke Malte Strasdat

Ethan Rublee

Gary R Bradski

Kurt Konolige

Stefan Hinterstoisser

Troy Straszheim

John William Zevenbergen

Steve Croft

Kevin William Watts

Kyle Lutz

Example embodiments may relate to methods and systems for selecting a grasp point on an object. In particular, a robotic manipulator may identify characteristics of a physical object within a physical environment. Based on the identified characteristics, the robotic manipulator may determine potential grasp points on the physical object corresponding to points at which a gripper attached to the robotic manipulator is operable to grip the physical object. Subsequently, the robotic manipulator may determine a motion path for the gripper to follow in order to move the physical object to a drop-off location for the physical object and then select a grasp point, from the potential grasp points, based on the determined motion path. After selecting the grasp point, the robotic manipulator may grip the physical object at the selected grasp point with the gripper and move the physical object through the determined motion path to the drop-off location.

Object pickup strategies for a robotic device

Methods and systems for using projected patterns to facilitate mapping of an environment are provided herein. A computing system may cause fixedly-posed projectors to each provide, onto a respective area of the environment, a predetermined respective distinct pattern. The system may determine respective poses of the projectors, and further determine a map of the environment that identifies, for each distinct pattern, respective locations on one or more surfaces in the environment on which the distinct pattern is detectable. Based on sensor data the system may identify a portion of a particular distinct pattern in the environment. The system may use the map and the respective pose of a particular projector that is providing the particular pattern to make a determination that the portion is located at a new, different location compared to the map. The system may then transmit an output signal indicating the determination.

Methods and systems for using projected patterns to facilitate mapping of an environment

Methods and systems for remote perception assistance to facilitate robotic object manipulation are provided herein. From a model of objects in an environment of a robotic manipulator, a virtual boundary line separating two adjacent identified virtual objects may be identified. The robotic manipulator may be configured to perform a task involving a manipulation of at least one object in the environment represented by the identified virtual objects. Based on the identifying, a request for remote assistance with verifying the virtual boundary line may be sent to a remote assistor device. A response to the request, including a modification to the virtual boundary line, may then be received from the remote assistor device. The robotic manipulator may then be caused to perform the task based on the modification to the virtual boundary line.

Methods and systems for remote perception assistance to facilitate robotic object manipulation

Example systems and methods allow for dynamic updating of a plan to move objects using a robotic device. One example method includes determining a virtual environment by one or more processors based on sensor data received from one or more sensors, the virtual environment representing a physical environment containing a plurality of physical objects, developing a plan, based on the virtual environment, to cause a robotic manipulator to move one or more of the physical objects in the physical environment, causing the robotic manipulator to perform a first action according to the plan, receiving updated sensor data from the one or more sensors after the robotic manipulator performs the first action, modifying the virtual environment based on the updated sensor data, determining one or more modifications to the plan based on the modified virtual environment, and causing the robotic manipulator to perform a second action according to the modified plan.

Continuous updating of plan for robotic object manipulation based on received sensor data

Methods and systems for detecting and reconstructing environments to facilitate robotic interaction with such environments are described. An example method may involve determining a three-dimensional (3D) virtual environment representative of a physical environment of the robotic manipulator including a plurality of 3D virtual objects corresponding to respective physical objects in the physical environment. The method may then involve determining two-dimensional (2D) images of the virtual environment including 2D depth maps. The method may then involve determining portions of the 2D images that correspond to a given one or more physical objects. The method may then involve determining, based on the portions and the 2D depth maps, 3D models corresponding to the portions. The method may then involve, based on the 3D models, selecting a physical object from the given one or more physical objects. The method may then involve providing an instruction to the robotic manipulator to move that object.

Growing community of inventors

San Francisco, CA, United States of America

Hauke Malte Strasdat