Technologies Holdings Corporation

Microsoft Technology Licensing, LLC

Mark Alan Foresman

Bradley J Prevost

Peter Willem Van Der Sluis

Marijn Van Anart

Robert Godfrey

Alireza Janani

Andrew James Pratt

Richard B Voigt

Monil Dinesh Morar

Marinus P Van Aart

Marijn Van Aart

Nolan Grant Moore

A leg () detection system comprising: a robotic arm () comprising a gripping portion () for holding a teat cup () for attaching to a teat (S, 203) of a dairy livestock (); an imaging system coupled to the robotic arm () and configured to capture a first three-dimensional (3D) image () of a rearview of the dairy livestock () in a stall (), the imaging system comprising a 3D camera () or a laser (), wherein each pixel of the first 3D image () is associated with a depth value; one or more memory () devices configured to store a reference (3D) 3D image () of the stall () without any dairy livestock (); and a processor () communicatively coupled to the imaging system and the one or more memory () devices, the processor () configured to: access the first 3D image () and the reference (3D) 3D image (); subtract the first 3D image () from the reference (3D) 3D image () to produce a second 3D image (); perform morphological image () processing on the second 3D image () to produce a third 3D image (); perform image () thresholding on the third 3D image () to produce a fourth 3D image (); cluster () data from the fourth 3D image (); identify, using the clustered data from the fourth 3D image (), one or more legs () of the dairy livestock (); and provide instructions for movements of the robotic arm () to avoid the identified one or more legs () while attaching the teat cup () to the teat (S,) of the dairy livestock ().

Vision system for leg detection

Systems and methods presented herein include an anti-collision and motion monitoring system includes light detection and ranging (LiDAR) systems configured to detect locations of objects in an environment. The anti-collision and motion monitoring system also includes camera systems configured to capture images of the objects in the environment that are detected by the LiDAR systems. The anti-collision and motion monitoring system further includes processing circuitry configured to coordinate operation of the LiDAR systems and the camera systems, to receive inputs from the LiDAR systems and the camera systems relating to the objects in the environment, to process the inputs received from the LiDAR systems and the camera systems to determine outputs relating to monitoring, control, and alerting relating to the objects in the environment, and to communicate the outputs to a central coordinator.

Anti-collision and motion monitoring, control, and alerting systems and methods

Systems and methods presented herein include an anti-collision and motion monitoring system includes one or more light detection and ranging (LiDAR) systems configured to detect locations of one or more objects in an environment. The anti-collision and motion monitoring system also includes one or more camera systems configured to capture images of the one or more objects in the environment that are detected by the one or more LiDAR systems. The anti-collision and motion monitoring system further includes processing circuitry configured to coordinate operation of the one or more LiDAR systems and the one or more camera systems, to receive inputs from the one or more LiDAR systems and the one or more camera systems relating to the one or more objects in the environment, to process the inputs received from the one or more LiDAR systems and the one or more camera systems to determine outputs relating to control of at least one of the one or more objects in the environment, and to communicate the outputs to a central coordinator to control one or more operating parameters of at least one of the one or more objects in the environment based at least in part on the inputs received from the one or more LiDAR systems and the one or more camera systems.

Anti-collision and motion control systems and methods

A vision system that includes a robotic arm and a three-dimensional (3D) camera operably coupled to a processor. The processor is configured to acquire a 3D image and identify a teat of the dairy livestock within the 3D image. The processor is further configured to determine a distance between a portion of a robotic arm and an approach vector for the teat, to compare the distance between the portion of the robotic arm and the approach vector for the teat and the attachment range threshold value, and to send instructions to the robotic arm based on the comparison.

Vision system with teat detection

A system that includes a robotic arm, a laser, a memory, and a processor. The processor is configured to position the laser adjacent to a dairy livestock and to modify teat location information for one or more teats of the dairy livestock based on a robot position offset between the dairy livestock and the robotic arm. The processor is further configured to generate a teat position associated with an unknown teat based on a scan of the dairy livestock and to determine a position distances between the teat position and teats of the dairy livestock. The processor is further configured to identify a teat of the dairy livestock with the smallest position distance, to associate a teat identifier for the unknown teat with the identified teat, and to store the association in the memory.

Vision system with teat identification

A system that includes a robotic arm, a laser, a three-dimensional (3D) camera, and a processor. The processor is configured to send instructions to position the robotic arm adjacent to a dairy livestock, to send a signal that initiates scanning a portion of the dairy livestock, and to generate teat candidate position information based on the scan of the dairy livestock. The processor is further configured receive target teat information, identify a teat candidate within a teat location range of the target teat, and link the identified teat candidate with a teat identifier. The processor is further configured to send instructions to the robotic arm to move at least a portion of the robotic arm toward the target teat based on the teat candidate position information for the identified teat candidate that is linked with the teat identifier.

Vision system with teat candidate identification

A vision system that includes a robotic arm and a three-dimensional (3D) camera operably coupled to a processor. The processor is configured to position the robotic arm adjacent to the dairy livestock and acquire a 3D image using the 3D camera. The processor is further configured to identify a set of teat candidates within the 3D image and to filter the set of teat candidates based on one or more filtering rules. The processor is further configured to determine an aggregate teat candidate score for the consolidated set of teat candidates, compare the aggregate teat candidate score to a score threshold value, and update teat location information for the dairy livestock in response to determining the aggregate teat candidate score is greater than or equal to the score threshold value.

A system includes a robotic arm, a laser, and a processor. The processor is configured to determine whether a distance between a left and right teat of a dairy livestock is less than or equal to a predetermined distance, and if so, command the robotic arm to move to a scan location that is between the left and right teats. The processor is further configured to command the laser to perform a scan of the teats after the robotic arm is at the scan location and to determine whether the left and right teats are found in the scan. If both the left and right teats are found in the scan, the processor commands the robotic arm to attach a teat cup to either the left or right teat.

Vision system for teat detection

A system includes a robotic arm, a laser, and a processor. The processor is configured to determine whether a first teat of a diary livestock is found in a first scan of the dairy livestock by the laser, and if so, command the robotic arm to move to a location corresponding to the location of the first teat. The processor is further configured to increment a counter associated with the first teat and then determine whether the counter equals a predetermined number. If the counter equals the predetermined number, the processor commands the robotic arm to attach a teat cup to the first teat. If the counter does not equal the predetermined number, the processor determines whether the first teat is found in a second scan by the laser. If the first teat is found in the second scan, the processor commands the robotic arm to attach the teat cup to the first teat.

A system that includes a laser configured to generate a profile signal of at least a portion of a dairy livestock and a processor. The processor is configured to obtain the profile signal and detect one or more edge pair candidates in the profile signal, compare the complementary distance gradients of each of the one or more edge pair candidates to a minimum distance gradient length to be considered an edge pair, and identify an edge pair from among the one or more edge pair candidates as a teat candidate based on the comparison. The processor is further configured to determine position information for the teat candidate.

A system includes a robotic arm, a laser, and a processor. The processor is configured to determine that a teat cup is to be attached to a front teat of a dairy livestock, and in response, determine an amount of separation between the front teat and a rear teat. The processor is further configured to calculate, if the amount of separation between the front and rear teats is greater than or equal to a predetermined distance, an updated front teat position based on the amount of separation between the front and rear teats and command the robotic arm to move to the updated front teat position. The processor is further configure to determine whether the front teat is found in a scan of the dairy livestock by the laser, and if so, command the robotic arm to attach the teat cup to the front teat.

A leg () detection system comprising: a robotic arm () comprising a gripping portion () for holding a teat cup () for attaching to a teat () of a diary livestock (); an imaging system coupled to the robotic arm () and configured to capture a first three-dimensional (3D) image () of a rearview of the dairy livestock () in a stall (), the imaging system comprising a 3D camera () or a laser (), wherein each pixel of the first 3D image () is associated with a depth value; one or more memory () devices configured to store a reference (3D) 3D image () of the stall () without any dairy livestock (); and a processor () communicatively coupled to the imaging system and the one or more memory () devices, the processor () configured to: access the first 3D image () and the reference (3D) 3D image (); subtract the first 3D image () from the reference (3D) 3D image () to produce a second 3D image (); perform morphological image () processing on the second 3D image () to produce a third 3D image (); perform image () thresholding on the third 3D image () to produce a fourth 3D image (); cluster () data from the fourth 3D image (); identify, using the clustered data from the fourth 3D image () to avoid the identified one or more legs () while attaching the teat cup () to the teat (S,) of the dairy livestock ().

A system includes a robotic arm, a laser, and a processor. The processor is configured to command the robotic arm to move to a first location corresponding to an expected teat position of a first teat. The processor is further configured to determine whether the first teat is found in a first scan by the laser, and if so, command the robotic arm to move to a second location corresponding to the location in the scan. The processor is further configured to determine whether the first teat is found in a second scan by the laser, and if so, determine whether the first teat is within a predetermined distance from a current location of the robotic arm. If the first teat is within the predetermined distance from the current location of the robotic arm, the processor commands the robotic arm to attach a teat cup to the first teat.

A system includes a robotic arm, memory, and a processor. The processor is configured to access first and second success counters stored in the memory. The first success counter is associated with a first attach algorithm and the second success counter is associated with a second attach algorithm. The processor is further configured to determine whether the first success counter is greater than the second success counter. The processor is further configured to execute the first attach algorithm if the first success counter is greater than the second success counter, and to execute the second attach algorithm if the first success counter is not greater than the second success counter. The processor is further configured to increment the first success counter if the first attach algorithm is successful, and to increment the second success counter if the second attach algorithm is successful.

A system includes a robotic arm, a laser, and a processor. The processor is configured to command the robotic arm to move to a location of an expected teat position of a dairy livestock and to command the laser to perform two scans of the dairy livestock. The processor is further configured to determine whether a first teat is found in both of the scans, and if so, determine whether locations of the first teat in the two scans are within a predetermined distance of each other. The processor is further configured to, in response to determining that locations of the first teat in the two scans are within the predetermined distance of each other, command the robotic arm to move to a location corresponding to the location of the first teat and to command the robotic arm to attach a teat cup to the first teat.

Growing community of inventors

Houston, TX, United States of America

Mark Alan Foresman