Crown Equipment Corporation

Other

Yoder Software, Inc.

John-David S Yoder

Michael J Seelinger

Edwin R Elston

Lee F Holeva

Eric L Jensen

Steven B Skaar

A method of tracking scored candidate objects in a sequence of image frames acquired by a camera is provided. The scored candidate objects may comprise a set of existing scored candidate objects associated with a prior image frame and a set of new scored candidate objects associated with a next image frame.

Identifying, matching and tracking multiple objects in a sequence of images

A method is provided for evaluating a possible pallet object in a gray scale image including one or more pallets. The method may comprise: generating, by a computer, a first confidence score Scorefor a lower left corner associated with the possible pallet object; generating, by the computer, a second confidence score Scorefor a lower right corner associated with the possible pallet object; generating, by the computer, a third confidence score Scorefor a bottom pallet board line associated with the possible pallet object; generating, by the computer, a fourth confidence score Scorefor a center stringer associated with the possible pallet object; and calculating, by the computer, a composite score Scorebased on the first, second, third and fourth confidence scores.

Generating a composite score for a possible pallet in an image scene

A method is provided for controlling forks of a vehicle. The method may comprise: while the forks of the vehicle are moving vertically, acquiring a series of images of a scene of a physical environment in which a plurality of pallets are visible; identifying in each image, by a computer system, one or more scored candidate objects, each potentially corresponding to a respective one of the plurality of pallets; for each of the one or more scored candidate objects, tracking, by the computer system, a respective location in each of at least two images of the series; determining for each of the one or more scored candidate objects a respective, associated height in the physical environment; and stopping, by the computer system, the forks of the vehicle at a height in the physical environment based on the height of a specific one of the scored candidate objects.

Controlling truck forks based on identifying and tracking multiple objects in an image scene

A method is provided for identifying one or more scored candidate objects that may correspond to one or more actual pallets in a gray scale image. The method may comprise: identifying, by a computer, a first plurality of scored candidate objects in the gray scale image; storing, by the computer, a list of the first plurality of scored candidate objects, wherein the list includes a respective record for each of the first plurality of scored candidate objects; determining, by the computer, a subset of the first plurality of scored candidate objects to eliminate from the list based on a comparison amongst the respective records of the first plurality of scored candidate objects; and removing, by the computer, the subset of the first plurality of scored candidate objects from the list to create an updated list for a second plurality of scored candidate objects, wherein the updated list includes a respective record for each of the second plurality of scored candidate objects.

Identifying and selecting objects that may correspond to pallets in an image scene

A method is provided for evaluating a possible pallet structure. The method comprises: providing a gray scale image comprising one or more possible lines; providing respective pixel locations in the gray scale image for an estimated upper left corner and an estimated upper right corner; calculating, using a computer, a value h based on the estimated upper left corner location and the estimated upper right corner location; estimating, using the computer, a first hole with a first rectangle having a height h; estimating, using the computer, a second hole with a second rectangle having the height h; and estimating, using the computer, the possible center stringer with a third rectangle having the height h.

Identifying and evaluating multiple rectangles that may correspond to a pallet in an image scene

A method is provided for identifying one or more scored candidate objects that may correspond to one or more actual pallets in a gray scale image. The method may comprise: identifying, by a computer, a first plurality of scored candidate objects in the gray scale image; storing, by the computer, a starting list of the first plurality of scored candidate objects, wherein the starting list includes a respective record for each of the first plurality of scored candidate objects that includes a respective location of the scored candidate object within the gray scale image, and a respective composite object score that represents a likelihood that the scored candidate object corresponds to an actual pallet structure in the gray scale image; and removing from the starting list each of the first plurality of scored candidate objects which: a) is located within a predetermined vertical distance of a particular one of the first plurality of scored candidate objects; and b) has a respective composite score less than that of the particular one scored candidate object.

Selecting objects within a vertical range of one another corresponding to pallets in an image scene

A programmable computer-implemented method is provided for finding possible corners of a pallet in an image. The method may comprise: acquiring a grey scale image including one or more pallets; determining, using a computer, horizontal cross correlations between the image and a first step-edge template to generate a set of horizontal cross correlation results; determining, using the computer, vertical cross correlations between the image and a second step-edge template to generate a set of vertical cross correlation results; and determining, using the computer, a first set of pixels, each such pixel respectively corresponding to a possible first corner of the one or more pallets, using a first corner template, the set of horizontal cross correlation results and the set of vertical cross correlation results.

Evaluating features in an image possibly corresponding to an intersection of a pallet stringer and a pallet board

A method is provided for evaluating a possible center stringer of a pallet. The method may comprise providing a first Ro image; providing a second Ro image comprising pixels that may generally correspond to an orthogonal distance from an origin point to one or more possible vertical right lines in the corresponding gray scale image; identifying, using a computer, a possible lower left corner location of a center stringer in the corresponding gray scale image; determining, using the computer, an upper left corner location based on the possible lower left corner location; identifying, using the computer, a possible lower right corner location of the center stringer in the corresponding gray scale image; and determining, using the computer, an upper right corner location based on the possible lower right corner location.

Identifying and evaluating potential center stringers of a pallet in an image scene

The present application provides a system for enabling instrument placement from distances on the order of five meters, for example, and increases accuracy of the instrument placement relative to visually-specified targets. The system provides precision control of a mobile base of a rover and onboard manipulators (e.g., robotic arms) relative to a visually-specified target using one or more sets of cameras. The system automatically compensates for wheel slippage and kinematic inaccuracy ensuring accurate placement (on the order of 2 mm, for example) of the instrument relative to the target. The system provides the ability for autonomous instrument placement by controlling both the base of the rover and the onboard manipulator using a single set of cameras. To extend the distance from which the placement can be completed to nearly five meters, target information may be transferred from navigation cameras (used for long-range) to front hazard cameras (used for positioning the manipulator).

Method and system for providing autonomous control of a platform

The invention is a method of using computer vision to control systems consisting of a combination of holonomic and nonholonomic degrees of freedom such as a wheeled rover equipped with a robotic arm, a forklift, and earth-moving equipment such as a backhoe or a front-loader. Using vision sensors mounted on the mobile system and the manipulator, the system establishes a relationship between the internal joint configuration of the holonomic degrees of freedom of the manipulator and the appearance of features on the manipulator in the reference frames of the vision sensors. Then, the system, perhaps with the assistance of an operator, identifies the locations of the target object in the reference frames of the vision sensors. Using this target information, along with the relationship described above, the system determines a suitable trajectory for the nonholonomic degrees of freedom of the base to follow towards the target object. The system also determines a suitable pose or series of poses for the holonomic degrees of freedom of the manipulator. With additional visual samples, the system automatically updates the trajectory and final pose of the manipulator so as to allow for greater precision in the overall final position of the system.

Growing community of inventors

Bluffton, OH, United States of America

John-David S Yoder