X Development LLC

Mineral Earth Sciences LLC

Deere + Company

Google Inc.

Zhiqiang Yuan

Elliott Grant

Jie Yang

Cheng-en Guo

Bodi Yuan

Kangkang Wang

Lianghao Li

Wilson Zhao

Hongxu Ma

Ming Zheng

Yueqi Li

Sergey Yaroshenko

Hongxiao Liu

Benoit M Schillings

Bin Ni

Matthew A Bitterman

William Raymond Regan

Qianyu Zhang

David R Brown

Yujing Qian

Francis Ebong

Matt VanCleave

Rhishikesh Pethe

Theodore Monyak

Cheng-En Guo

Techniques are disclosed herein that enable generating a relationship embedding indicating a relationship between one or more agricultural attributes of a table of agricultural data with one or more nodes in an agricultural knowledge graph. Various implementations include processing a table of agricultural data with rows of agricultural records and columns of agricultural attributes. Additional or alternative implementations include processing the table of agricultural data using an embedding model portion of the mapping model to generate an embedding space representation of each of the agricultural attributes. Various implementations can include selecting a node corresponding to a given agricultural attribute based on a distance between the embedding space representation of the given agricultural attribute and the embedding space representations of one or more nodes.

Ingesting schema-less agricultural data for machine learning pipeline(s)

Implementations set forth herein relate to utilizing S2 cell values to characterize arbitrary portions of land parcels and storing the S2 cell values in association with a non-fungible token (NFT) that is stored on a blockchain network, or other peer-to-peer (P2P) network. The S2 cell values can be generated by iteratively using bounding shapes that are selected to extend over at least a portion of a respective parcel of land, and each bounding shape can be represented by one or more single dimensional values. When a generated bounding shape extends outside of a boundary of a parcel of land, subcells of the bounding shape can be generated to define further bounding shapes. A land NFT for the list of cell values for the bounding shapes can be stored at a blockchain address for an authenticated owner of the parcel of land.

Associating land NFTs with digital representations of land parcels

Implementations are described herein for colorizing an X-ray image and predicting one or more phenotypic traits about a plant based on the colorized X-ray image. In various implementations, an X-ray image that depicts a plant with a canopy of the plant partially occluding a part-of-interest is obtained, where the part-of-interest is visible through the canopy in the X-ray image. The X-ray images is colorized to predict one or more phenotypic traits of the part-of-interest. The colorization includes processing the X-ray image based on a machine learning model to generate a colorized version of the X-ray image, and predicting the one or more phenotypic traits based on one or more visual features of the colorized version of the X-ray image.

Colorizing x-ray images

Techniques are disclosed that enable generating a predicted yield for a cereal grain crop based on one or more traits extracted from image(s) of the cereal grain crop. Various implementations include determining a heading trait value based on the number of identified spikelets, where the spikelets are identified by processing the image(s) of the cereal grain crop using a spikelet detection model. Additional or alternative implementations include generating a predicted cereal grain crop yield based on one or more additional or alternative trait values such as one or more heading values, one or more projected leaf area values, one or more stand spacing values, one or more wheat rust values, one or more maturity detection values, one or more intercropping phenotyping values extracted cereal grains intercropped with other crops, one or more additional or alternative trait output values, and/or combinations thereof.

Determining cereal grain crop yield based on cereal grain trait value(s)

Implementations are described herein for conditioning a generator machine learning model to generate synthetic ground-level data that is biased towards a given agricultural area based on high-elevation images. In various implementations, a plurality of ground-level images may be accessed that depict crops within a specific agricultural area. A first set of high-elevation image(s) may also be accessed that depict the specific agricultural area. The ground-level images and the first set of high-elevation image(s) may be used to condition an air-to-ground generator machine learning model to generate synthetic ground-level data from high-elevation imagery depicting the specific agricultural area. A second set of high-elevation image(s) that depict a specific sub-region of the specific agricultural area may then be accessed and processed using the air-to-ground generator machine learning model to generate synthetic ground-level data that infers one or more conditions of the specific sub-region of the specific agricultural area.

Generating synthetic ground-level data based on generator conditioned for particular agricultural area

Implementations are directed to generating a stream of agricultural annotations with respect to area(s) of interest of an agricultural field, and providing the stream of agricultural annotations for presentation to the user in an augmented reality manner with respect to the area(s) of interest. In some implementations, a stream of vision data may be received at a first computing device of the user and from a second computing device of the user. Further, the first computing device may process the stream of vision data to generate the stream of agricultural annotations. Moreover, the first computing device may transmit the stream of agricultural annotations to the second computing device to cause the stream of agricultural annotations to be provided for presentation to the user. In other implementations, the first computing device may be omitted, and the second computing device may be utilized to generate the stream of agricultural annotations.

Method(s) and system(s) for enabling augmented reality scouting of an agricultural field

Implementations are described herein for incorporating unstructured data into machine learning-based phenotyping. In various implementations, natural language textual snippet(s) may be obtained. Each natural language textual snippet may describe environmental or managerial features of an agricultural plot that exist during a crop cycle. A sequence-to-sequence machine learning model may be used to encode the natural language snippet(s) into embedding(s) in embedding space. The embedding(s) may semantically represent the environmental or managerial features of the agricultural plot. Using one or more phenotypic machine learning models, phenotypic prediction(s) may be generated about the agricultural plot based on the one or more semantic embeddings and additional structured data about the agricultural plot. Output may be provided at one or more computing devices that is based on one or more of the phenotypic predictions.

Incorporating unstructured data into machine learning-based phenotyping

Implementations are described herein for training and applying machine learning models to digital images capturing plants, and to other data indicative of attributes of individual plants captured in the digital images, to recognize individual plants in distinction from other individual plants. In various implementations, a digital image that captures a first plant of a plurality of plants may be applied, along with additional data indicative of an additional attribute of the first plant observed when the digital image was taken, as input across a machine learning model to generate output. Based on the output, an association may be stored in memory, e.g., of a database, between the digital image that captures the first plant and one or more previously-captured digital images of the first plant.

Individual plant recognition and localization

Implementations are disclosed for adaptively reallocating computing resources of resource-constrained devices between tasks performed in situ by those resource-constrained devices. In various implementations, while the resource-constrained device is transported through an agricultural area, computing resource usage of the resource-constrained device ma may be monitored. Additionally, phenotypic output generated by one or more phenotypic tasks performed onboard the resource-constrained device may be monitored. Based on the monitored computing resource usage and the monitored phenotypic output, a state may be generated and processed based on a policy model to generate a probability distribution over a plurality of candidate reallocation actions. Based on the probability distribution, candidate reallocation action(s) may be selected and performed to reallocate at least some computing resources between a first phenotypic task of the one or more phenotypic tasks and a different task while the resource-constrained device is transported through the agricultural area.

Adaptively reallocating resources of resource-constrained devices

Techniques are described herein for using artificial intelligence to predict crop yields based on observational crop data. A method includes: obtaining a first digital image of at least one plant; segmenting the first digital image of the at least one plant to identify at least one seedpod in the first digital image; for each of the at least one seedpod in the first digital image: determining a color of the seedpod; determining a number of seeds in the seedpod; inferring, using one or more machine learning models, a moisture content of the seedpod based on the color of the seedpod; and estimating, based on the moisture content of the seedpod and the number of seeds in the seedpod, a weight of the seedpod; and predicting a crop yield based on the moisture content and the weight of each of the at least one seedpod.

Inferring moisture from color

Implementations are described herein for localizing individual plants by aligning high-elevation images using invariant anchor points while disregarding variant feature points, such as deformable plants. High-elevation images that capture the plurality of plants at a resolution at which wind-triggered deformation of individual plants is perceptible between the high-elevation images may be obtained. First regions of the high-elevation images that depict the plurality of plants may be classified as variant features that are unusable as invariant anchor points. Second regions of the high-elevation images that are disjoint from the first set of regions may be classified as invariant anchor points. The high-elevation images may be aligned based on invariant anchor point(s) that are common among at least some of the high-elevation images. Based on the aligned high-elevation images, individual plant(s) may be localized within one of the high-elevation images for performance of one or more agricultural tasks.

Localization of individual plants based on high-elevation imagery

Implementations are described herein for localizing individual plants using high-elevation images at multiple different resolutions. A first set of high-elevation images that capture the plurality of plants at a first resolution may be analyzed to classify a set of pixels as invariant anchor points. High-elevation images of the first set may be aligned with each other based on the invariant anchor points that are common among at least some of the first set of high-elevation images. A mapping may be generated between pixels of the aligned high-elevation images of the first set and spatially-corresponding pixels of a second set of higher-resolution high-elevation images. Based at least in part on the mapping, individual plant(s) of the plurality of plants may be localized within one or more of the second set of high-elevation images for performance of one or more agricultural tasks.

Implementations are described herein for coordinating semi-autonomous robots to perform agricultural tasks on a plurality of plants with minimal human intervention. In various implementations, a plurality of robots may be deployed to perform a respective plurality of agricultural tasks. Each agricultural task may be associated with a respective plant of a plurality of plants, and each plant may have been previously designated as a target for one of the agricultural tasks. It may be determined that a given robot has reached an individual plant associated with the respective agricultural task that was assigned to the given robot. Based at least in part on that determination, a manual control interface may be provided at output component(s) of a computing device in network communication with the given robot. The manual control interface may be operable to manually control the given robot to perform the respective agricultural task.

Coordinating agricultural robots

Implementations are disclosed for selectively operating edge-based sensors and/or computational resources under circumstances dictated by observation of targeted plant trait(s) to generate targeted agricultural inferences. In various implementations, triage data may be acquired at a first level of detail from a sensor of an edge computing node carried through an agricultural field. The triage data may be locally processed at the edge using machine learning model(s) to detect targeted plant trait(s) exhibited by plant(s) in the field. Based on the detected plant trait(s), a region of interest (ROI) may be established in the field. Targeted inference data may be acquired at a second, greater level of detail from the sensor while the sensor is carried through the ROI. The targeted inference data may be locally processed at the edge using one or more of the machine learning models to make a targeted inference about plants within the ROI.

Edge-based processing of agricultural data

Implementations are described herein for auditing performance of large-scale tasks. In various implementations, one or more ground-level vision sensors may capture a first set of one or more images that depict an agricultural plot prior to an agricultural task being performed in the agricultural plot, and a second set of one or more images that depict the agricultural plot subsequent to the agricultural task being performed in the agricultural plot. The first and second sets of images may be processed in situ using edge computing device(s) based on a machine learning model to generate respective pluralities of pre-task and post-task inferences about the agricultural plot. Performance of the agricultural task may include comparing the pre-task inferences to the post-task inferences to generate operational metric(s) about the performance of the agricultural task in the agricultural plot. The operational metric(s) may be presented at one or more output devices.

Growing community of inventors

San Jose, CA, United States of America

Zhiqiang Yuan