Deere & Company

X Development LLC

Mineral Earth Sciences LLC

Yueqi Li

Jie Yang

Nanzhu Wang

Elliott Grant

Zhiqiang Yuan

Alexander Ngai

Cheng-en Guo

Kathleen Watson

Chunfeng Wen

Hongxu Ma

Sergey Yaroshenko

Alan Eneev

Yujing Qian

Hongxiao Liu

Erich Schlaepfer

Shuhao Fu

Sicong Wang

Gaoxiang Chen

Various implementations include processing an instance of image data using a reinforcement learning policy model to generate illumination output, where the illumination output indicates one or more lights of an agricultural robot or modular sensor package to adjust based on uneven illumination in the instance of image data. In many implementations, the initial instance of image data is captured using one or more sensors of an agricultural robot or modular sensor package, where the initial instance of image data captures one or more crops in a portion of a plot of land. In various implementations, the agricultural robot or modular sensor package can adjust one or more lights based on the illumination output, and can capture an updated instance of image data of the given one or more crops with the updated illumination.

Dynamic lighting for plant imaging

Implementations are described herein for utilizing the spectral-, spatial-, and temporal-information of a satellite image time series to facilitate crop control or monitoring. In various implementations, a plurality of training examples may be assembled for a generative model. Each training example of the plurality of training examples may include a respective high-resolution image capturing a respective region and a corresponding low-resolution satellite image time series capturing the respective region. The plurality of training examples can be used to train the generative model, to acquire a trained generative model. A high-resolution image depicting one or more agricultural conditions for a given region, can be received and processed using the trained generative model, to generate a synthetic low-resolution satellite image time series, where the synthetic low-resolution satellite image time series represent the one or more agricultural conditions.

Time series generator trained using satellite data

Implementations are disclosed for automatically generating computer code that implements a machine learning-based processing pipeline based on multiple different modalities of input. In various implementations, one or more annotations created on a demonstration digital image to annotate one or more visual features depicted in the demonstration digital image may be processed to generate annotation embedding(s). Natural language input describing one or more operations to be performed based on the one or more annotations also may be processed to generate one or more logic embeddings. The annotation embedding(s) and the logic embedding(s) may be processed using a language model to generate, and store in non-transitory computer-readable memory, target computer code. The target computer code may implement a machine learning-based processing pipeline that performs the one or more operations based on the one or more annotations.

Generating machine learning pipelines using natural language and/or visual annotations

Implementations are described herein for training a remote sensing machine learning model. In various implementations, ground-truth low elevation images that depict particular crop(s) in particular agricultural area(s) and terrain conditions observed in the agricultural area(s) are identified. A plurality of low elevation training images is generated based on the ground truth low elevation training images to include the plurality of ground truth low elevation images and synthetic low elevation images generated based on synthetic terrain conditions. The plurality of low-elevation training images are processed using a synthetic satellite image machine learning model to generate the plurality of synthetic satellite training images, which are then processed by the remote sensing machine learning model to generate inferred terrain conditions. The remote sensing machine learning model is then trained based on a comparison of the inferred terrain conditions and the corresponding observed or synthetic terrain conditions.

Generating synthetic training data and training a remote sensing machine learning model

Implementations are disclosed for facilitating visual programming of machine learning state machines. In various implementations, one or more graphical user interfaces (GUIs) may be rendered on one or more displays. Each GUI may include a working canvas on which a plurality of graphical elements corresponding to at least some of a plurality of available logical routines are manipulable to define a machine learning state machine. One or more of the available logical routines may include logical operations that process data using machine learning model(s). Two or more at least partially redundant logical routines that include overlapping logical operations may be identified, and overlapping logical operations of the two or more at least partially redundant logical routines may be merged into a consolidated logical routine. At least some of the logical operations that were previously downstream from the overlapping logical operations may be logically coupled with the consolidated logical routine.

Visual programming of machine learning state machines

Implementations are described herein for training and applying machine learning models to digital images capturing plants, and to other data indicative of attributes of individual plants captured in the digital images, to recognize individual plants in distinction from other individual plants. In various implementations, a digital image that captures a first plant of a plurality of plants may be applied, along with additional data indicative of an additional attribute of the first plant observed when the digital image was taken, as input across a machine learning model to generate output. Based on the output, an association may be stored in memory, e.g., of a database, between the digital image that captures the first plant and one or more previously-captured digital images of the first plant.

Individual plant recognition and localization

Implementations are disclosed for dynamically allocating aspects of platform-independent machine-learning based agricultural state machines among edge and cloud computing resources. In various implementations, a GUI may include a working canvas on which graphical elements corresponding to platform-independent logical routines are manipulable to define a platform-independent agricultural state machine. Some of the platform-independent logical routines may include logical operations that process agricultural data using phenotyping machine learning model(s). Edge computing resource(s) available to a user for which the agricultural state machine is to be implemented may be identified. Constraint(s) imposed by the user on implementation of the agricultural state machine may be ascertained. Based on the edge computing resource(s) and constraint(s), logical operations of some platform-independent logical routines may be dynamically allocated to the edge computing resource(s), and logical operations of other platform-independent logical routines may be dynamically allocated to a cloud computing resource.

Dynamic allocation of platform-independent machine learning state machines between edge-based and cloud-based computing resources

Implementations are described herein for learning mappings between a domain specific language (DSL) and images, and leveraging those mappings for various purposes. In various implementations, a method for using a DSL to generate training data may include processing data indicative of ground truth image(s) depicting a real plant using a trained image-to-DSL machine learning (ML) model to generate a first expression in the DSL that describes structure of the real plant. The first expression may include a plurality of parameters, and may be processed to programmatically generate a plurality of synthetic DSL expressions. Each respective synthetic DSL expression may describe structure of a respective synthetic plant for which parameter(s) have been altered from the first expression. The synthetic DSL expressions may be processed using a renderer to create three-dimensional (3D) synthetic plant models. Two-dimensional (2D) synthetic images may be generated that depict the 3D synthetic plant models in an area.

Generating domain specific language expressions based on images

Implementations are described herein for reducing the time and costs associated with the crop scouting in a crop field. In various implementations, a method is implemented using one or more processors, and the method include: operating, based on a type and arrangement of a crop field, a robot to travel along a trajectory through the crop field using a first gait. The robot includes one or more vision sensors. The first gait includes a first repeating cycle of poses of the robot. The method can further include: synchronizing operation of one or more of the vision sensors with one or more poses of the first repeating cycle of poses of the multi-legged robot to capture one or more initial sequences of images depicting one or more points-of-interest of crops growing in the crop field.

Adaptive scouting using multi-legged robots

Implementations are described herein for edge-based real time crop yield predictions made using sampled subsets of robotically-acquired vision data. In various implementations, one or more robots may be deployed amongst a plurality of plants in an area such as a field. Using one or more vision sensors of the one or more robots, a superset of high resolution images may be acquired that depict the plurality of plants. A subset of multiple high resolution images may then be sampled from the superset of high resolution images. Data indicative of the subset of high resolution images may be applied as input across a machine learning model, with or without additional data, to generate output indicative of a real time crop yield prediction.

Edge-based crop yield prediction

Techniques are disclosed that enable generating a unified data set by mapping a set of item description phrases, describing entries in a data set, to a set of canonical phrases. Various implementations include generating a similarity measure between each item description phrase and each canonical phrase by processing the corresponding item description phrase and the corresponding canonical phrase using a natural language processing model. Additional or alternative implementations include generating a bipartite graph based on the set of item description phrases, the set of canonical phrases, and the similarity measures. The mapping can be generated based on the bipartite graph.

Mapping data set(s) to canonical phrases using natural language processing model(s)

Implementations are directed to assigning corresponding semantic identifiers to a plurality of rows of an agricultural field, generating a local mapping of the agricultural field that includes the plurality of rows of the agricultural field, and subsequently utilizing the local mapping in performance of one or more agricultural operations. In some implementations, the local mapping can be generated based on overhead vision data that captures at least a portion of the agricultural field. In these implementations, the local mapping can be generated based on GPS data associated with the portion of the agricultural field captured in the overhead vision data. In other implementations, the local mapping can be generated based on driving data generated during an episode of locomotion of a vehicle through the agricultural field. In these implementations, the local mapping can be generated based on GPS data associated with the vehicle traversing through the agricultural field.

Generating a local mapping of an agricultural field for use in performance of agricultural operation(s)

Implementations are described herein for using machine learning to determine whether candidate crop fields are suitable for management by particular agricultural entities. In various implementations, a machine learning model may be applied to input data to generate output data. The input data may include a first plurality of data points corresponding to field-level agricultural management practices of an agricultural entity. The output data may be indicative of one or more predicted outcomes of the agricultural entity implementing the field-level agricultural management practices on one or more candidate crop fields not currently managed by the agricultural entity. Based on one or more of the predicted outcomes, one or more computing devices may be caused to provide a user associated with the agricultural entity with information about one or more of the candidate crop fields, and/or one or more parameter inputs of a graphical user interface may be prepopulated.

Growing community of inventors

San Jose, CA, United States of America

Yueqi Li