The Climate Corporation

Wei Guan

Pramithus Khadka

Ying She

Yichuan Gui

Jeffrey Gerard

Yaqi Chen

Ying Xu

Demir Devecigil

Xiaoyuan Yang

Marian Farah

Daniel Williams

Juan Pablo Bedoya

Victor Stuber

Gerard Guillemette

Joost Kemink

Julian Boshard

Erik Andrejko

Gardar Johannesson

Systems and methods for identifying clouds and cloud shadows in satellite imagery are described herein. In an embodiment, a system receives a plurality of images of agronomic fields produced using one or more frequency bands. The system also receives corresponding data identifying cloud and cloud shadow locations in the images. The system trains. a machine learning system to identify at least cloud locations using the images as inputs and at least data identifying pixels as cloud pixels or non-cloud pixels as outputs. When the system receives one or more particular images of a particular agronomic field produced using the one or more frequency bands, the system uses the one or more particular images as inputs into the machine learning system to identify a plurality of pixels in the one or more particular images as particular cloud locations.

Machine learning techniques for identifying clouds and cloud shadows in satellite imagery

A computer system is provided comprising a classification model management server computer configured, by instructions, to: receive a new image from a user device; apply a first digital model to first regions within the new image for classifying each of the first regions into a particular class; apply a second digital model to second regions within the new image for classifying each of the second regions into a particular class; and transmit classification data related to the class of the first regions and the class of the second regions to the user device. In connection therewith, the second regions each generally correspond to a combination of multiple first regions.

Detection of plant diseases with multi-stage, multi-scale deep learning

A system and processing methods for configuring and utilizing a convolutional neural network (CNN) for plant disease recognition are disclosed. In some embodiments, the system is programmed to collect photos of infected plants or leaves where regions showing symptoms of infecting diseases are marked. Each photo may have multiple marked regions. Depending on how the symptoms are sized or clustered, one marked region may include only one lesion caused by one disease, while another may include multiple, closely-spaced lesions caused by one disease. The system is programmed to determine anchor boxes having distinct aspect ratios from these marked regions for each convolutional layer of a single shot multibox detector (SSD). For certain types of plants, common diseases lead to relatively many aspect ratios, some having relatively extreme values. The system is programmed to then train the SSD using the marked regions and the anchor boxes and apply the SSD to new photos to identify diseased plants.

Detecting infection of plant diseases by classifying plant photos

A system for detecting clouds and cloud shadows is described. In one approach, clouds and cloud shadows within a remote sensing image are detected through a three step process. In the first stage a high-precision low-recall classifier is used to identify cloud seed pixels within the image. In the second stage, a low-precision high-recall classifier is used to identify potential cloud pixels within the image. Additionally, in the second stage, the cloud seed pixels are grown into the potential cloud pixels to identify clusters of pixels which have a high likelihood of representing clouds. In the third stage, a geometric technique is used to determine pixels which likely represent shadows cast by the clouds identified in the second stage. The clouds identified in the second stage and the shadows identified in the third stage are then exported as a cloud mask and shadow mask of the remote sensing image.

Cloud detection on remote sensing imagery

In some embodiments, a computer-implemented method is disclosed. The method comprises receiving a plant image from a user device and applying a first digital model to first regions within the image for classifying each of the first regions into a class of a first set of classes corresponding to a first plurality of plant diseases, a healthy condition, or a combination of a second plurality of plant diseases. The method also includes applying a second digital model to one or more second regions within the image for classifying each of the one or more second regions into a class of a second set of classes corresponding to the second plurality of plant diseases. The method then includes transmitting classification data related to the classes of the first set of classes and the classes of the second set of classes to the user device into which the regions are classified.

In an embodiment, digital images of agricultural fields are received at an agricultural intelligence processing system. Each digital image includes a set of pixels having pixel values, and each pixel value of a pixel includes a plurality of spectral band intensity values. Each spectral band intensity value describes a spectral band intensity of one band among several bands of electromagnetic radiation. For each of the agricultural fields, spectral band intensity values of each band are preprocessed at a field level using the digital images for that agricultural field resulting in preprocessed intensity values. The preprocessed intensity values are provided as input to a machine learning model. The model generates a predicted yield value for each field. The predicted yield value is used to update field yield maps of agricultural fields for forecasting and can be displayed via a graphical user interface (GUI) of a client computing device.

In-season field level yield forecasting

A system for identifying ponding water located on a field from image data is described. In an approach, an image of an agricultural field is analyzed using a classifier that has been trained based on the spectral bands of labeled image pixels to identify a probability for each pixel within the image that the pixel corresponds to water. A flow simulation is performed to determine regions of the field that are likely to pool water after rainfall based on precipitation data, elevation data, and soil property data of the field. A graph of vertices representing the pixels and edges representing connections between neighboring pixels is generated. The probability of each pixel within the graph being ponding water is set based on the probability pixel being water, the likelihood that water will pool in the area represented by the pixel, the probability of neighboring pixels being ponding water, and a cropland mask that identifies which pixels correspond to cropland. A class for each pixel is then determined that maximizes the joint probability over the graph.

Ponding water detection on satellite imagery

Systems and methods for identifying clouds and cloud shadows in satellite imagery are described herein. In an embodiment, a system receives a plurality of images of agronomic fields produced using one or more frequency bands. The system also receives corresponding data identifying cloud and cloud shadow locations in the images. The system trains a machine learning system to identify at least cloud locations using the images as inputs and at least data identifying pixels as cloud pixels or non-cloud pixels as outputs. When the system receives one or more particular images of a particular agronomic field produced using the one or more frequency bands, the system uses the one or more particular images as inputs into the machine learning system to identify a plurality of pixels in the one or more particular images as particular cloud locations.

In some embodiments, a computer-implemented method is disclosed. The method comprises obtaining a first digital model for classifying an image into a class of a first set of classes corresponding to a first plurality of plant diseases, a healthy condition, or a combination of a second plurality of plant diseases; obtaining a second digital model for classifying an image into a class of a second set of classes corresponding to the second plurality of plant diseases; receiving a new image from a user device; applying the first digital model to a plurality of first regions within the new image to obtain a plurality of classifications; applying the second digital model to one or more second regions, each corresponding to a combination of multiple first regions of the plurality of first regions, to obtain one or more classifications, the multiple first regions being classified into the class corresponding to the combination of the second plurality of plant diseases; transmitting classification data related to the plurality of classifications into a class corresponding to one of the first plurality of plant diseases or the healthy condition and the one or more classifications to the user device.

In an embodiment, a computer-implemented method is disclosed. The method comprises causing a camera to continuously capture surroundings to generate multiple images and causing a display device to continuously display the multiple images as the multiple images are generated. In addition, the method comprises processing each of one or more of the multiple images. The processing comprises identifying at least one of a plurality of diseases and calculating at least one disease score associated with the at least one disease for a particular image; causing the display device to display information regarding the at least one disease and the at least one disease score in association with a currently displayed image; receiving input specifying one or more of the at least one disease; and causing the display device to show additional data regarding the one or more diseases, including a remedial measure for the one or more diseases.

Crop component count

In an approach, hyperspectral and/or multispectral remote sensing images are automatically analyzed by a nitrogen analysis subsystem to estimate the value of nitrogen variables of crops or other plant life located within the images. For example, the nitrogen analysis subsystem may contain a data collector module, a function generator module, and a nitrogen estimator module. The data collector module prepares training data which is used by the function generator module to train a mapping function. The mapping function is then used by the nitrogen estimator module to estimate the values of nitrogen variables for a new remote sensing image that is not included in the training set. The values may then be reported and/or used to determine an optimal amount of fertilizer to add to a field of crops to promote plant growth.

Estimating nitrogen content using hyperspectral and multispectral images

A system and processing methods for refining a convolutional neural network (CNN) to capture characterizing features of different classes are disclosed. In some embodiments, the system is programmed to start with the filters in one of the last few convolutional layers of the initial CNN, which often correspond to more class-specific features, rank them to hone in on more relevant filters, and update the initial CNN by turning off the less relevant filters in that one convolutional layer. The result is often a more generalized CNN that is rid of certain filters that do not help characterize the classes.

Growing community of inventors

Fremont, CA, United States of America

Wei Guan