Nonholonomic camera space manipulation

Abstract

A nonholonomic camera space manipulation system which allows a mobile manipulator which can have both holonomic and nonholonomic movement to autonomously use computer vision to move with respect to a goal position without any prior knowledge or calibration between the two or more video cameras used with the computer vision and the manipulator base or arm, the cameras and the goal, or the base and arm and the goal position. Cues are associated with the manipulator arm and the goal position(s) or target bodies; the cues being distinguishable in the two dimensional focal plane camera spaces of the cameras from the surrounding environment. A processing unit, identifying the position of the cues in each camera space, compares those positions and instructs movements of the manipulator base and/or manipulator arm to achieve movement of the visual cue on the arm with respect to the goal position or target body visual cue in camera space. These instructions are based upon estimations of what movement is required to bring two visual cues in each camera space together or into the required camera-space configuration. Once movement of the manipulator base and arm has begun, the system tracks this movement and adjusts its estimations and instructions according to monitoring of the visual cues in the camera spaces. By utilizing the different angles of the two or more camera means, the continuous monitoring of the visual cue positions in each camera space leads