Caterpillar Inc.

Perkins Engines Company Limited

Peter Joseph Petrany

Jacob Charles Maley

Douglas Jay Husted

Vamsi Krishna Pannala

Robert Scott McFarland

Benjamin J Hodel

David Lee Zwetz, Ii

Raymond Alan Wise

Shastri Ram

Rodrigo L Sanchez

Simon Richard Conway

Samuel Alec Roberts

Nicholas Payne

Matthew A Guttag

Thomas Henry Abel

Michael Flatman

Robert S Mcfarland

Hassen Ali

Edward Yii-Hsin Lin

Adam Paul Roesner

Jeremy Lee Vogel

Pranay Kumar Reddy Kontham

A wear detection system can be configured to receive a video stream including a plurality of images of a bucket of the work machine from a camera associated with the work machine. The bucket has one or more ground engaging tools (GET). The wear detection system can also be configured to identify a plurality of tool images from the video stream over a period of time. The plurality of tool images depict the GET at a plurality of instances over a period of time. The wear detection system can also be configured to determine a plurality of tool pixel counts from the plurality of tool image and determine a wear level for the GET based on the plurality of tool pixel counts.

Ground engaging tool wear and loss detection system and method

An object detection vision system and methods are disclosed. A method for detecting objects in a vision system of an industrial machine includes receiving image data from one or more vision cameras and receiving detection data from one or more detect devices. The detection data includes one or more detected objects. The method includes combining the detection data and the image data and transforming the detection data in the image data based on one or more objects in the image data. The method also includes displaying an indication of the detected one or more objects in the image data based on the transformed detection data.

Object detection vision system

Changes in ambient light intensity are detected by processing image data from an image capture device to determine, iteratively, a signal-to-noise ratio of the image data. The signal-to-noise ratio is a ratio of average to variance of pixel values for some of the pixels forming the image. A control output is generated, based on the signal-to-noise ratio, that is responsive to changes in ambient light intensity. The control output is used control a light source of a vehicle.

Vehicle lighting control using image processing of changes in ambient light intensity

The present disclosure is directed to systems and methods for estimating machine rotation state. The rotation state can include the rotation speed and/or angle as the machine rotates. In some implementations, an estimating machine rotation state system can receive image data, timestamped consecutively, from camera devices of a machine. The estimating machine rotation state system can generate optical flow vectors for the pixels of the image data and determine the hue and intensity of the pixels based on the optical flow vectors. Using the hues, intensities, and timestamps of the image data, the estimating machine rotation state system can generate an intensity plot over time. The estimating rotation states system can then determine a rotation speed of the machine based on the intensity plot. The estimating rotation states system can also determine a rotation angle of the machine by integrating the intensity plot over time.

Estimating the moving state of rotating machines systems and methods

A system and method for monitoring objects in the vicinity of a work machine that is in motion includes obtaining, using a camera associated with the work machine, first image data of a field-of-view of the camera at a first position; identifying an object-of-interest within the first image data; associating an alarm with the object-of-interest, the alarm configured to indicate the presence of the object-of-interest; presenting the alarm through an output device; receiving an input to snooze the alarm for the object-of-interest; generating a mask for the object-of-interest using image coordinates of the object-of-interest within the first image data; obtaining, using the camera, second image data of the field-of-view of the camera at a second position during motion of the work machine; identifying an object within the second image data; and selectively aborting the alarm presentation through the output device in response to determining, using the mask, that the object is the object-of-interest.

Object tracking for work machines

An object detection system may include an imager configured to generate image data indicative of an environment in which a machine is present, and a sensor configured to generate sensor data indicative of the environment in which the machine is present. The object detection system may further include an object detection controller including one or more object detection processors configured to receive an image signal indicative of the image data, identify an object associated with the image data, and determine a first location of the object relative to the position of the imager. The one or more object detection processors may also be configured to receive a sensor signal indicative of the sensor data, and determine, based at least in part on the sensor signal, the presence or absence of the object at the first location.

System and method for detecting objects

A method receiving image information with one or more processor(s) and from a sensor disposed at a worksite and determining an identity of a work tool disposed at the worksite based at least partly on the image information. The method further includes receiving location information with the one or more processor(s), the location information indicating a first location of the sensor at the worksite. Additionally, the method includes determining a second location of the work tool at the worksite based at least partly on the location information. In some instances, the method includes generating a worksite map with the one or more processor(s), the worksite map identifying the work tool and indicating the second location of the work tool at the worksite, and at least one of providing the worksite map to an additional processor and causing the worksite map to be rendered via a display.

System and method for managing tools at a worksite

A controller may process a plurality of video frames to determine an apparent motion of each pixel of one or more pixels or each group of pixels of one or more groups of pixels of each video frame of the plurality of video frames. The controller may select one or more processed video frames, of the plurality of processed video frames, that correspond to a duration of time and may generate a composite video frame based on the one or more processed video frames. The controller may generate a video frame mask based on the composite video frame and may obtain additional video data that includes at least one additional video frame. The controller may cause the video frame mask to be applied to the at least one additional video frame and may cause the at least one additional video frame to be processed using an object detection technique.

Excluding a component of a work machine from a video frame based on motion information

A method, for monitoring an implement of a machine, includes receiving implement parameters associated with the implement and determining a range of movement of the implement based on the implement parameters. The method detects whether at least one object is located within the range of movement of the implement and displays a visual representation of the range of movement with a visual indication indicative of whether the at least one object is located within the range of movement. The method detects changes to the implement parameters and determines an updated range of movement based on the detected changes. Further, the method determines whether at least one object is located within the updated range of movement and displays a visual representation of the updated range of movement with an updated visual indication indicative of whether the at least one object is located within the updated range of movement.

Method and system for monitoring a rotatable implement of a machine

A surround view system for a machine is provided. The surround view system includes a plurality of image capturing devices generating image data of surroundings of the machine, and an object detection system for detecting an object in a target field of view of the machine and generating object position data corresponding to the object. The surround view system also includes an image processing system configured to generate an initial 3-dimensional composite surround view by projecting the image data on a virtual model corresponding to the machine. The virtual model has a 3-dimensional shape based on an initial calibration position data. The image processing system is also configured to modify the 3-dimensional shape of the virtual model based on the object position data, and generate an updated 3-dimensional composite surround view by projecting the image data on the virtual model having a modified 3-dimensional shape.

Machine surround view system and method for generating 3-dimensional composite surround view using same

An imaging system for a machine having an implement includes at least two cameras mounted on the machine at opposing sides of the implement and configured to generate at least two camera views of an environment of the machine, and a processing device in communication with the at least two cameras. The processing device is configured to obtain the at least two camera views, obtain at least one of a position or an angle of the implement, determine at least one of a position or an angle of a stitching line according to the at least one of the position or the angle of the implement, and generate a stitched surround view based on the at least two camera views and the stitching line.

Imaging system for generating a surround-view image

A vision system for a machine system includes a first set of cameras mounted on the first section and a second set of cameras mounted on the second section of the machine. A controller is coupled to the first set of cameras and the second set of cameras. The controller includes a first and a second image transformation modules to convert a first and a second images captured by the first and the second set of cameras into a first and a second transformed image based at least on a view type, respectively. An image combination module combines the first transformed image and the second transformed image to a combined image with reference to the articulation joint. The image combination module communicates with a display device to display the combined image. The combined image is indicative of surroundings of the first section and the second section of the machine.

Vision system and method of monitoring surroundings of machine

An obstacle detection system includes an object detection system, an alert system, and a controller. The controller is configured to store an electronic map containing a known obstacle, store known and unknown obstacle warning zones, and detect a detected object in proximity to the machine. The controller is further configured to compare the detected object to the known obstacle to determine whether the detected object is a known obstacle or an unknown obstacle, generate a first alert signal if the detected object is a known obstacle and within the known obstacle warning zone, and generate a second alert signal if the detected object is an unknown obstacle and within the unknown obstacle warning zone.

Growing community of inventors

Dunlap, IL, United States of America

Peter Joseph Petrany