Microsoft Technology Licensing, LLC

Bryan Joseph Thibodeau

Harshavardhana Narayana Kikkeri

Mihai R Jalobeanu

Michael David Revow

Grigor Shirakyan

Harshavardhana Kikkeri

Michael Revow

Mihai Jalobeanu

A method, system, and one or more computer-readable storage media for controlling a robot in the presence of a moving object are provided herein. The method includes capturing a number of frames from a three-dimensional camera system and analyzing a frame to identify a connected object. The frame is compared to a previous frame to identify a moving connected object (MCO). If an unexpected MCO is in the frame a determination is made if the unexpected MCO is in an actionable region. If so, the robot is instructed to take an action.

Controlling a robot in the presence of a moving object

Various technologies described herein pertain to automatic in-situ calibration and registration of a depth sensor and a robotic arm, where the depth sensor and the robotic arm operate in a workspace. The robotic arm can include an end effector. A non-parametric technique for registration between the depth sensor and the robotic arm can be implemented. The registration technique can utilize a sparse sampling of the workspace (e.g., collected during calibration or recalibration). A point cloud can be formed over calibration points and interpolation can be performed within the point cloud to map coordinates in a sensor coordinate frame to coordinates in an arm coordinate frame. Such technique can automatically incorporate intrinsic sensor parameters into transformations between the depth sensor and the robotic arm. Accordingly, an explicit model of intrinsics or biases of the depth sensor need not be utilized.

Automatic in-situ registration and calibration of robotic arm/sensor/workspace system

Data about a physical object in a real-world environment is automatically collected and labeled. A mechanical device is used to maneuver the object into different poses within a three-dimensional workspace in the real-world environment. While the object is in each different pose an image of the object is input from one or more sensors and data specifying the pose is input from the mechanical device. The image of the object input from each of the sensors for each different pose is labeled with the data specifying the pose and with information identifying the object. A database for the object that includes these labeled images can be generated. The labeled images can also be used to train a detector and classifier to detect and recognize the object when it is in an environment that is similar to the real-world environment.

Automated collection and labeling of object data

Growing community of inventors

Sammamish, WA, United States of America

Bryan Joseph Thibodeau