Amazon Technologies, Inc.

Emilio Ian Maldonado

Nishitkumar Ashokkumar Desai

Daniel Bibireata

Sudarshan Narasimha Raghavan

Ammar Chinoy

Yasser Baseer Asmi

Hao Jiang

David Allen Smith

Jaechul Kim

Xiaofeng Ren

Frank Florian Liberato, Jr

Pranab Mohanty

Subhrangshu Nandi

Dilip Kumar

Prithviraj Banerjee

Pahal Kamlesh Dalal

Muralidhar Koka

Shrivatsan Vasudhevan

Joachim Sebastian Stahl

Krzysztof Marcin Walczak

Yi Lei

Alice Helen Swanberg

Bella Dubrov

Min Xu

Vishal Rathi

Kevin Kar Wai Lai

Wisam Hussain

Alexey Korobko

Hussain Badruddin Penwala

Described is a multiple-camera system and process for determining an item involved in an event. For example, when a user picks an item or places an item at an inventory location, image information for the item may be obtained and processed to identify the item involved in the event and associate that item with the user.

Store tracking system

Described is a system and method for collecting item and user information and utilizing that information to determine materials handling facility patterns, item trends and inventory location trends. User monitoring data for users located in a materials handling facility may include information identifying inventory locations approached by users, gaze directions of users, user dwell times, an identification of items picked by users, an identification of items placed by users, and/or in-transit times for items. This information may be aggregated and processed to determine materials handling facility patterns, item trends and/or inventory location trends.

Determining trends from materials handling facility information

Type ambiguity detection methods and systems are provided to predict a browse node classification for items in an electronic catalog and determine ambiguity between browse nodes based at least partly on the predicted browse node classifications for the items. For example, the type ambiguity detection system may identify a plurality of search queries previously submitted by users of the electronic catalog, determine a plurality of browse nodes that are frequently selected after submitting the search queries, determine items that consumers frequently view or purchase in connection with the search queries, input the items into a classifier, receive an output predicting a proper item type or browse node for a given item, and initiate an action in response to determining ambiguity.

Type ambiguity detection system

Determining an item involved in an event at an event location

Sensors in a facility generate sensor data associated with a region of the facility, which can be used to determine a 3D location of an object in the facility. Some sensors may sense overlapping regions of the facility. For example, a first sensor may generate data associated with a first region of the facility, while a second sensor may generate data associated with a second region of the facility that partially overlaps the first region. Sensors may fail at times as determined from sensor output data or status data. In response to identifying a failed sensor, an undetected region corresponding to the failed sensor is identified, as well as a substitute sensor that partially senses the undetected region. Sensor data from the substitute sensor, such as 2D data, is acquired and used to estimate a 3D location of an object in the undetected region.

Management of sensor failure in a facility

Systems and methods are provided for learning item types of items listed in an electronic repository, and for training a machine learning model to predict the item type of a given input item. For example, a machine learning model may be obtained or accessed that has been previously trained to classify an input item to a browse node. Vector representations of individual items assigned to different browse nodes may be obtained from an intermediate layer of the previously trained machine learning model, and a vector representation of individual browse nodes may then be generated based on the vector representations of individual items assigned to that browse node. A clustering algorithm may be applied to the browse node vector representations in order to identify clusters of similar browse nodes, where individual clusters may represent different unique item types.

Item type discovery and classification using machine learning

Type completeness detection methods and systems are provided to determine a browse node completeness score for browse nodes in an electronic catalog. For example, the type completeness detection system may select a browse node, identify a plurality of search queries previously submitted by users of the electronic catalog, determine a plurality of items that are frequently selected after submitting the search queries, calculate a query completeness score for each search query, determine a browse node completeness score for the browse node, and initiate an action in response to determining that the browse node completeness score falls below a threshold value.

Type completeness detection system

Described is a multiple-camera system and process for detecting a user within a materials handling facility and tracking a position of the user as the user moves through the materials handling facility. In one implementation, a plurality of depth sensing cameras are positioned above a surface of the materials handling facility and oriented to obtain a top-down view of the surface of the materials handling facility, along with any objects (e.g., users) on the surface of the materials handling facility. The depth information from the cameras may be utilized to detect objects on the surface of the materials handling facility, track a movement of those objects and determine if those objects are users.

Collecting user pattern descriptors for use in tracking a movement of a user within a materials handling facility

During operation, a facility may utilize many sensors, such as cameras, to generate sensor data. The sensor data may be processed by an inventory management system to track objects, determine the occurrence of events at the facility, and so forth. At any given time, some of these sensors may fail to provide timely data, may fail to provide any data, may generate inaccurate data, and so forth. Described are techniques to determine failure of sensors and adjust operation of the inventory management system to maintain operability during sensor failure.

Described is a multiple-camera system and process for re-identifying a user located in a materials handling facility based on user patterns and/or descriptors representative of the user. In one implementation, a user pattern and/or a plurality of descriptors representative of a user are maintained as a position of a user is tracked through a materials handling facility. If the tracking of the user is lost, the last known position is stored with the user pattern and/or descriptors. If a new object is detected and confirmed to be a user, a user pattern and/or descriptors of the new object are compared with the stored user pattern and/or descriptors to determine if the new object is the user.

Segmenting a user pattern into descriptor regions for tracking and re-establishing tracking of a user within a materials handling facility

Described is a multiple-camera system and process for disambiguating between multiple users and identifying which of the multiple users performed an event. For example, when an event is detected, user patterns near the location of the event are determined, along with touch points at the location of the event. User pattern orientation and/or arm trajectories between the event location and the user patterns may be determined and processed to disambiguate between multiple users and determine which user pattern is involved in the event.

Disambiguating between multiple users

Described is a multiple-camera system and process for identifying a user that performed an event and associating that user with the event. For example, when an event is detected, user patterns near the location of the event are determined, along with touch points at the location of the event. User pattern orientation and/or arm trajectories between the event location and the user pattern may be determined and processed to link the user pattern to the event, thereby confirming the association between the event and the user.

Associating a user with an event

Determining an item involved in an event

Described is a multiple-camera system for use in capturing images of users within a materials handling facility and processing those images to monitor the movement of users. For large materials handling facilities, a large number of cameras may be required to monitor the facility. Processing of the data generated from a large number of cameras becomes difficult. The implementations described herein include a hierarchy that allows image data from any number of cameras within a materials handling facility to be processed without substantially increasing the processing time needed or sacrificing processing capabilities.

Growing community of inventors

Seattle, WA, United States of America

Emilio Ian Maldonado