Amazon Technologies, Inc.

Other

Oded Maron

Jeremy Samuel De Bonet

Ejaz Ahmed

Mirko Ristivojevic

Jonathan Evan Cohn

Daniel Ron

Xiang Ma

Kilho Son

Nicholas McMahon

Sreemanananth Sadanand

Sajjad Beygiharchegani

Michael Derryberry

Tomer Anor

Shamil Abzalov

This disclosure is directed to item-identifying carts that may be utilized by users in material handling facilities to automatically identify items that the users place in their carts as they move around the material handling facilities. In addition, these carts may automatically determine the outcome of respective events that occur with respect to these identified items. For example, the carts may be configured to identify one or more items that are placed into or removed from the cart, and thereafter determine one or more actions taken with respect to the identified items and a quantity of the items involved.

Resolving events in item-identifying carts

This disclosure is directed to, in part, a processing pipeline for detecting predefined activity using image data, identifying an item-of-interest and a location of the item-of-interest across frames of the image data, determining a trajectory of the item-of-interest, and determining an identifier of the item-of-interest and an action taken with respect to the item-of-interest. The processing pipeline may utilize one or more trained classifiers and, in some instances, additional data to identify items that are placed into or removed from a tote (e.g., basket, cart, or other receptacle) by users in material handling facilities as the users move around the material handling facilities.

Using sensor data to determine activity

This disclosure is directed to using cascading algorithms to automatically identify items placed in a tote or other receptacle utilized by users in material handling facilities as the users move around the facilities. A tote may store a database or 'gallery' of item representations for all of the items that are stored in the facility that a user may place in their totes. The tote may use multiple algorithms in a cascading manner to analyze the gallery of item representations in order to iteratively narrow the search space of item representations in the gallery to determine which of the items was placed in the tote by a user. Upon identifying the item placed in the tote, the tote may add an item identifier for the item to a virtual listing of item identifiers representing items previously placed in the tote.

Identifying items using cascading algorithms

This disclosure is directed to item-identifying carts that may be utilized by users to automatically identify items that the users place in their carts. In addition, these carts may automatically determine the outcome of respective events that occur with respect to these identified items. For example, the carts may be configured to identify one or more items that are placed into or removed from the cart, and thereafter determine one or more actions taken with respect to the identified items and a quantity of the items involved. For example, after identifying a first item and a second item either placed into or removed from the cart, the cart may determine that the user added two instances of the first item and removed one instance of the second item. In response to making this determination, the cart may update a virtual cart of a user operating the physical cart.

This disclosure is directed to item-identifying carts that may be utilized by users to automatically identify items that the users place in their carts. In order to identify the items placed into or removed from the cart, the cart may analyze image data, as well as weight data indicating a current weight estimation of a basket of the cart. The cart may include a weight sensor that generates a signal comprising a series of weight measurements of the basket of the cart. The cart may use an algorithm that calculates different mean values of different window sizes of the weight measurements of this signal and, thereafter, may calculate an average of these mean values. This average may be stored as the current weight estimation of the basket.

Efficient weight estimation and stabilization

This disclosure is directed to item-identifying carts that may generate and store image data of items being placed into, or removed from, the carts in the event that a cart is unable to recognize an item. For example, the cart may analyze frames of generated image data, representing item(s) being placed into or removed from the cart, to determine which one or more frames to store. If the cart is later unable to identify such an item, the cart may retrieve one or more of the stored frames, generate a user interface (UI) that includes these frame(s) and a request that a user operating the cart to help identify the item, and present the UI on a display of the cart.

A tag with sufficient variability is generated. The tag facilitates quick location and processing within a facility. During generation, a matrix representative of a possible tag may be tested for sufficient variability. A two-dimensional matrix representing the possible tag may include different bit values at different positions. A variability metric of the matrix is performed to check the variability between adjacent elements of the matrix. This variability metric of the matrix is then compared against a variability threshold value. If the variability metric of the matrix falls below the threshold value, the matrix is flagged as unusable. If the variability metric of the matrix exceeds the threshold value, the matrix can be used to generate a tag with sufficient variability. Once generated, the tag can be presented and affixed to an item or object to help identify and locate the item within the facility.

System for generating tags

This disclosure is directed to an item-identifying cart that may be utilized by a user in a materials handling facility to automatically identify items that the user places in their cart, and update a virtual shopping cart to include items taken by the user. The mobile cart may include four capture assemblies that are disposed proximate to each of the four corners of a basket of the cart, and oriented such that their respective optical axes are directed towards an interior of a perimeter of the top of the basket, and above the top of the basket. The capture assemblies may include proximity sensors that are used to detect movement above the top of the basket, LEDs that illuminate items, and cameras that generate image data representing the items as they are placed in, or removed from, the cart.

Item-identifying carts

This disclosure describes using trained classifiers and additional data to identify items placed into a tote. The techniques may use a first classifier on image data representing an item to determine a region associated with a barcode of the item and may use one or more second classifiers to analyze the region for determining a candidate barcode. The techniques may then determine whether the candidate barcode corresponds to the actual barcode of the item. If so, the techniques may update a virtual cart associated with a user account to indicate the acquisition of the item. If not, then the techniques may use the additional data for identifying the item. For example, the techniques determine a current location of a tote for determining items that are within a threshold distance of the location and may use this information, along with the candidate barcode, to identify the item placed into the tote.

Identifying item barcodes

A tag comprises cells represented by marks, such as white and black, arranged in a predetermined pattern. Reference cells are used to determine the edges of the tag and orientation, while others are used to encode data and error correction data. During processing, a tag is detected based on a variability metric within a given region of an image exceeding a threshold value. Once detected, the orientation of the tag is determined using the reference cells. Confidence values, indicative of a likelihood that the cell is a particular cell value, such as white or black, are determined for each cell in a first set of cell values. If the first set is invalid, the cell values for a set of lowest confidence cell values are inverted to produce a second set of cell values. The second set is tested, and if valid, is used to produce output data.

Growing community of inventors

Sudbury, MA, United States of America

Oded Maron