Blue River Technology Inc.

Deere + Company

James Patrick Ostrowski

Lee Kamp Redden

Edward William Jeffcott Pell

Kent M Anderson

Chia-Chun Fu

Christopher Grant Padwick

John William Peake

Rajesh Radhakrishnan

Olgert Denas

Benjamin Ray Chostner

Kent Michael Anderson

Andrei Polzounov

Chris Padwick

Thomas W Brown

Erik Ehn

Deaglan Halligan

Jacqueline Gao Song

James Willis

Divya Sharma

Bryon William Majusiak

Zeb Wheeler

Zebulon Elias Wheeler

A method including: recording a first image of a first field region; automatically treating a plant within the first region in-situ based on the first image; automatically verifying the plant treatment with a second image of the first region; and automatically treating a second region concurrently with treatment verification.

System and method for automated odometry calibration for precision agriculture systems

A farming machine including a number of treatment mechanisms treats plants according to a treatment plan as the farming machine moves through the field. The control system of the farming machine executes a plant identification model configured to identify plants in the field for treatment. The control system generates a treatment map identifying which treatment mechanisms to actuate to treat the plants in the field. To generate a treatment map, the farming machine captures an image of plants, processes the image to identify plants, and generates a treatment map. The plant identification model can be a convolutional neural network having an input layer, an identification layer, and an output layer. The input layer has the dimensionality of the image, the identification layer has a greatly reduced dimensionality, and the output layer has the dimensionality of the treatment mechanisms.

Semantic segmentation to identify and treat plants in a field and verify the plant treatments

The calibration system of the farming machine receives images from the camera array. The images comprise visual information representing a view of a portion of an area surrounding the farming machine. To calibrate the camera array, the system determines a relative pose between pairs of cameras by extracting relative position and orientation characteristics from visual information in images captured by the camera pairs. The calibration system determines a calibration error in part by propagating the relative poses between camera pairs. The calibration system may perform automated self-calibration by adjusting one or more of the cameras in the camera array, or may transmit remedial instructions to an operator to adjust the one or more cameras in the camera array.

Camera array calibration in a farming machine

A vehicle moves through an environment (e.g., a farming, construction, mining, or forestry environment) and performs one or more actions in the environment. Portions of the environment may include moisture, such as puddles or mud patches. A control system associated with the vehicle may include a traversability model or a moisture model to help the vehicle operate in the environment with the moisture. In particular, the control system may employ the traversability model to reduce the likelihood of the vehicle attempting to traverse an untraversable portion of the environment, and the control system may employ the moisture model to reduce the likelihood of the vehicle performing an action that will damage a portion of the environment.

Detecting untraversable environment and preventing damage by a vehicle

As a farming machine travels through a field of plants, the farming machine operates in a normal operational state to perform one or more farming operations. The farming machine detects an operational failure of a component of the farming machine using measurements obtained from one or more sensors coupled to and monitoring the farming machine. The operational failure of the component impacts performance of a first farming operation of the farming operations. The farming machine configures the farming machine to operate in a remedial operational state. In the remedial operational state, the farming machine diagnoses the operational failure of the component using the obtained measurements. In the remedial operational state, the farming machine selects a solution operation to address the operational failure of the component based on the diagnosis. The farming machine performs the determined solution operation.

Compensatory actions for automated farming machine failure

A farming machine includes one or more image sensors for capturing an image as the farming machine moves through the field. A control system accesses an image captured by the one or more sensors and identifies a distance value associated with each pixel of the image. The distance value corresponds to a distance between a point and an object that the pixel represents. The control system classifies pixels in the image as crop, plant, ground, etc. based on depth information in in the pixels. The control system generates a labelled point cloud using the labels and depth information, and identifies features about the crops, plants, ground, etc. in the point cloud. The control system generates treatment actions based on any of the depth information, visual information, point cloud, and feature values. The control system actuates a treatment mechanism based on the classified pixels.

Identifying and avoiding obstructions using depth information in a single image

A farming machine includes one or more image sensors for capturing an image as the farming machine moves through the field. A control system accesses the image(s) and creates a labelled three-dimensional point cloud representing the field. The control system identifies and treats plants based on the labelled point cloud. To do so, the control system applies pre-processing functions to the labelled point cloud to determine characteristics of the field and/or modify labels in the point cloud. Point clusters in the point cloud are identified as plants, crops, weeds, ground, etc., using the determined characteristics and modified labels. The control system derives feature values for the plants based on the determined characteristics and labels. If the feature value indicates that the plant should be treated, the farming machine actuates a treatment mechanism to treat the plant.

Extracting feature values from point clouds to generate plant treatments

The calibration system of the farming machine receives images from each camera of the camera array. The images comprise visual information representing a view of a portion of an area surrounding the farming machine. To calibrate a pair of cameras including a first camera and second camera, the calibration system determines a relative pose between the pair of cameras by extracting relative position and orientation characteristics from visual information in both an image received from the first camera and an image received from the second camera. The calibration system identifies a calibration error for the pair of cameras based on a comparison of the relative pose with an expected pose between the first pair of cameras. The calibration system transmits a notification to an operator of the farming machine that describes the calibration error and instructions for remedying the calibration error.

A farming machine identifies and treats a plant as the farming machine travels through a field. The farming machine includes a pair of image sensors for capturing images of a plant. The image sensors are different, and their output images are used to generate a depth map to improve the plant identification process. A control system identifies a plant using the depth map. The control system captures images, identifies a plant, and actuates a treatment mechanism in real time.

Plant identification using heterogenous multi-spectral stereo imaging

A plant treatment platform uses a plant detection model to detect plants as the plant treatment platform travels through a field. The plant treatment platform receives image data from a camera that captures images of plants (e.g., crops or weeds) growing in the field. The plant treatment platform applies pre-processing functions to the image data to prepare the image data for processing by the plant detection model. For example, the plant treatment platform may reformat the image data, adjust the resolution or aspect ratio, or crop the image data. The plant treatment platform applies the plant detection model to the pre-processed image data to generate bounding boxes for the plants. The plant treatment platform then can apply treatment to the plants based on the output of the machine-learned model.

Automated plant detection using image data

A modular system includes a hub and a set of modules removably coupled to the hub. The modules are physically coupled to the frame relative to each other so that each module can operate with respect to a different row of a field. An individual module includes a sensor for capturing field measurement data of individual plants along a row as the modular system moves through the geographic region. An individual module further includes a treatment mechanism for applying a treatment to the individual plants of the row based on the field measurement data before the modular system passes by the individual plants. An individual module further includes a computing device that determines the treatment based on the field measurement data and communicates data to the hub. The hub is communicatively coupled to the modules, so that it may exchange data between the modules and with a remote computing system.

Growing community of inventors

Mountain View, CA, United States of America

James Patrick Ostrowski