Amazon Technologies, Inc.

Other

Building Machines, Inc.

Jeremy Samuel De Bonet

Oded Maron

Jacob Alex Siegel

Nicholas McMahon

Ejaz Ahmed

Jonathan Evan Cohn

Mirko Ristivojevic

Thomas Duester

Brett Allen Skaloud

James Merton Tidman

Daniel Ron

Jeffrey Kevin Feiereisen

Dawn Marie Pfund

Michael Nathan Danenberg

Xiang Ma

Nathan Pius O'Neill

Kilho Son

Nikolai Orlov

Matthew Clark Webster

Nicholas Franklin

Robert P Irwin

Charles H Wood

Nicholas Mendenhall

Zihui Zhang

Sajjad Beygiharchegani

Sydney Tye Minnis

Somasundaram Niranjayan

Camerin Cole Hahn

Paul Eugene Munger

James Stephanick

Aaron M McDaniel

Jennifer Li

Robert M Riggs

Tomer Anor

Dylan Thomas Cox

Sreemanananth Sadanand

Patrick Ian McCleary

Nicholas Charles McMahon

Venkata Srikiran Bodapati

James Olney

Michele Pratusevich

This disclosure is directed to, in part, a processing pipeline for detecting predefined activity using image data, identifying an item-of-interest and a location of the item-of-interest across frames of the image data, determining a trajectory of the item-of-interest, and determining an identifier of the item-of-interest and an action taken with respect to the item-of-interest. The processing pipeline may utilize one or more trained classifiers and, in some instances, additional data to identify items that are placed into or removed from a tote (e.g., basket, cart, or other receptacle) by users in material handling facilities as the users move around the material handling facilities.

Using sensor data to determine activity

Embodiments of positioning systems and methods are disclosed herein. Embodiments of such positioning systems may include a hierarchy of positioning systems. Each of the positioning systems in the hierarchy may be adapted to move each positioning system lower in the hierarchy along with one or more end-effectors. A control system may control the positioning systems of the hierarchy using a control method comprising a coarse step, a refinement step, or an adaptive step.

Systems and methods for precise and dynamic positioning over volumes

This disclosure is directed to using cascading algorithms to automatically identify items placed in a tote or other receptacle utilized by users in material handling facilities as the users move around the facilities. A tote may store a database or 'gallery' of item representations for all of the items that are stored in the facility that a user may place in their totes. The tote may use multiple algorithms in a cascading manner to analyze the gallery of item representations in order to iteratively narrow the search space of item representations in the gallery to determine which of the items was placed in the tote by a user. Upon identifying the item placed in the tote, the tote may add an item identifier for the item to a virtual listing of item identifiers representing items previously placed in the tote.

Identifying items using cascading algorithms

This disclosure is directed to item-identifying carts that may be utilized by users to automatically identify items that the users place in their carts. In addition, these carts may automatically determine the outcome of respective events that occur with respect to these identified items. For example, the carts may be configured to identify one or more items that are placed into or removed from the cart, and thereafter determine one or more actions taken with respect to the identified items and a quantity of the items involved. For example, after identifying a first item and a second item either placed into or removed from the cart, the cart may determine that the user added two instances of the first item and removed one instance of the second item. In response to making this determination, the cart may update a virtual cart of a user operating the physical cart.

Resolving events in item-identifying carts

This disclosure is directed to an item-identifying, mobile cart that may be utilized by a user in a materials handling facility to automatically identify a user operating the cart and items that the user places into a basket of the cart. In addition, the cart may update a virtual shopping cart of the identified user to include items taken by the user. The mobile cart may include multiple imaging devices and oriented such that their respective optical axes are directed towards an interior of a perimeter of the top of the basket, and above the top of the basket. The mobile cart may also include an imaging device oriented away from the basket such that a user operating the mobile cart may scan a user identifier using this imaging device to enable recognition of the user.

Item-identifying carts

This disclosure is directed to item-identifying carts that may be utilized by users to automatically identify items that the users place in their carts. In order to identify the items placed into or removed from the cart, the cart may analyze image data, as well as weight data indicating a current weight estimation of a basket of the cart. The cart may include a weight sensor that generates a signal comprising a series of weight measurements of the basket of the cart. The cart may use an algorithm that calculates different mean values of different window sizes of the weight measurements of this signal and, thereafter, may calculate an average of these mean values. This average may be stored as the current weight estimation of the basket.

Efficient weight estimation and stabilization

A tag with sufficient variability is generated. The tag facilitates quick location and processing within a facility. During generation, a matrix representative of a possible tag may be tested for sufficient variability. A two-dimensional matrix representing the possible tag may include different bit values at different positions. A variability metric of the matrix is performed to check the variability between adjacent elements of the matrix. This variability metric of the matrix is then compared against a variability threshold value. If the variability metric of the matrix falls below the threshold value, the matrix is flagged as unusable. If the variability metric of the matrix exceeds the threshold value, the matrix can be used to generate a tag with sufficient variability. Once generated, the tag can be presented and affixed to an item or object to help identify and locate the item within the facility.

System for generating tags

A system that can create an approximated model from a trained machine learning model (such as a neural network) where the approximated model can operate using fewer computing resources than the original trained model. The system can create the approximated model without the voluminous training data used to create the original trained model. The system can rely on just the data describing the trained model and an indicator as to how closely the approximated model should correspond to the original model and/or the desired savings of computing resources. Various lossless and/or lossy approximations may be performed to obtain multiple approximated models that may be substituted for the trained model during runtime operations to achieve significant speed/cost savings over operation of the original trained model.

Trained model approximation

This disclosure is directed to an item-identifying cart that may be utilized by a user in a materials handling facility to automatically identify items that the user places in their cart, and update a virtual shopping cart to include items taken by the user. The mobile cart may include four capture assemblies that are disposed proximate to each of the four corners of a basket of the cart, and oriented such that their respective optical axes are directed towards an interior of a perimeter of the top of the basket, and above the top of the basket. The capture assemblies may include proximity sensors that are used to detect movement above the top of the basket, LEDs that illuminate items, and cameras that generate image data representing the items as they are placed in, or removed from, the cart.

Items may be stored on a fixture of a facility using one or more accessories, such as trays or bins. Weighing modules that include one or more load cells may be mounted to a crossbar of the fixture. The accessories of the fixture may then be mounted to the crossbar via the weighing modules. The load cells within each weighing module can collect weight data associated with items stowed in the accessories that are coupled to the weighing module. When a weight change is detected at the fixture, a particular load cell is identified as the source of the weight change. Once a load cell is identified, the accessory coupled to the load cell is identified, as well as the types of items stowed therein. Interaction data can then be determined to indicate a type of item that was added to or removed from the accessory of the fixture.

Detection of load cells associated with weight changes at a fixture

This disclosure describes using trained classifiers and additional data to identify items placed into a tote. The techniques may use a first classifier on image data representing an item to determine a region associated with a barcode of the item and may use one or more second classifiers to analyze the region for determining a candidate barcode. The techniques may then determine whether the candidate barcode corresponds to the actual barcode of the item. If so, the techniques may update a virtual cart associated with a user account to indicate the acquisition of the item. If not, then the techniques may use the additional data for identifying the item. For example, the techniques determine a current location of a tote for determining items that are within a threshold distance of the location and may use this information, along with the candidate barcode, to identify the item placed into the tote.

Identifying item barcodes

Growing community of inventors

Southborough, MA, United States of America

Jeremy Samuel De Bonet