Amazon Technologies, Inc.

Sudarshan Narasimha Raghavan

Yasser Baseer Asmi

Ammar Chinoy

Ohil Krishnamurthy Manyam

Nishitkumar Ashokkumar Desai

Dilip Kumar

Gopi Prashanth Gopal

Xiaofeng Ren

Avishkar Misra

Emilio Ian Maldonado

Daniel Bibireata

Liefeng Bo

Christopher Robert Towers

Ramanathan Palaniappan

Michel Leonard Goldstein

Shrivatsan Vasudhevan

Jin Dong Kim

Hao Jiang

Korwin Jon Smith

Jason Michael Famularo

Thomas Meilandt Mathiesen

Maren Marie Costa

Casey Louis Thurston

Christopher Richard Fescoe

Waqas Syed Ahmed

Danny Guan

Long Xuan Nguyen

Muralidhar Koka

Fan Sun

Nadya Dhalla

Sudheer Babu Ramoji

David William Bettis

Sridhar Boyapati

Alexander Michael McNamara

Aaron Craig Thompson

Alexander Clark Prater

Christopher Andrew Stephens

Félix Joseph Étienne Pageau

David Echevarria Ignacio

David Allen Smith

Frank Florian Liberato, Jr

Kevin Alexander Lee

Gary Paolo Raden

Pahal Kamlesh Dalal

Jaechul Kim

Pranab Mohanty

Zachary T Hubert

Min Xu

Kevin Kar Wai Lai

Pradeep Kancharia

Leon I Sverntlof

Described is a system for counting stacked items using image analysis. In one implementation, an image of an inventory location with stacked items is obtained and processed to determine the number of items stacked at the inventory location. In some instances, the item closest to the camera that obtains the image may be the only item viewable in the image. Using image analysis, such as depth mapping or Histogram of Oriented Gradients (HOG) algorithms, the distance of the item from the camera and the shelf of the inventory location can be determined. Using this information, and known dimension information for the item, a count of the number of items stacked at an inventory location may be determined.

Determining inventory changes at an inventory location

This disclosure describes a system for disambiguating between multiple potential users that may have performed an item action (e.g., item removal or item placement) at an inventory location. For example, if there are three picking agents (users) standing near an inventory location and one of the agents removes an item (item action) from the inventory location, the example systems and processes described herein may utilize various inputs to disambiguate between the users and determine which of the potential users performed the item action.

Disambiguating between users

Described is a multiple-camera system and process for determining an item involved in an event. For example, when a user picks an item or places an item at an inventory location, image information for the item may be obtained and processed to identify the item involved in the event and associate that item with the user.

Store tracking system

This disclosure describes an item drop location for placing items that have been picked from an inventory location within a material handling facility but have not yet been transitioned from the materials handling facility and a return location for returning items that have been transitioned from the materials handling facility. Likewise, described is a system and method for identifying an item placed at a drop location or an item placed at a return location, processing the placed item and providing confirmation to a user that placed the item and facilitating a return of the item or a removal of the item from an item identifier list associated with the user.

Inventory item release apparatus and method

Described is a method for determining when to transition devices between different states based on determined potential activity in an area near the devices. When the potential for activity near an input device is below a threshold, the device is maintained in an inactive state. When the potential for activity near the device exceeds the threshold, the device is transitioned to an active state before the potential activity can occur near the device. Likewise, when the device is in an active state, the implementations described herein provide the ability to determine when to begin and end transmission of data from the device to a remote computing resource for processing. For example, obtained video may be processed locally to determine if an activity is occurring. If an activity is occurring, relevant data is sent to a remote computing resource to determine if an action has been performed during the activity.

Selective transmission of data

A system may use sensor data from a facility to generate tentative values associated with an event, such as the identification of an item removed from a shelf of the facility. A confidence value associated with each of the tentative values may be less than a confidence threshold. In response, inquiry data seeking confirmation of a tentative value from an associate is generated and sent to one or more associates in the facility. Responses from the associates are collected to determine a selection of one of the tentative values. The selected tentative value is designated as output data for the system. Thereafter, the output data and the original sensor data are designated as training data, which can then be used to train or update machine learning systems. Subsequent use of the updated machine learning systems can yield more accurate results.

Updating machine learning systems using previous output data

Described is a system and method for collecting item and user information and utilizing that information to determine materials handling facility patterns, item trends and inventory location trends. User monitoring data for users located in a materials handling facility may include information identifying inventory locations approached by users, gaze directions of users, user dwell times, an identification of items picked by users, an identification of items placed by users, and/or in-transit times for items. This information may be aggregated and processed to determine materials handling facility patterns, item trends and/or inventory location trends.

Determining trends from materials handling facility information

A system may generate image data of users within a facility. Such users may be part of a group that is unknown to the system. The system can predict group data for two or more users based on a resemblance or the users being within a threshold distance of each other. However, if a confidence level associated with the predicted group data is below a threshold value, the group data is deemed unreliable and assistance from an associate is deemed necessary. A user interface that includes a portion of the image data, information about the predicted group data, and other interface elements is presented to the associate via a display. Based on the input data received from the associate, the group data can be confirmed or rejected. If the group data is confirmed, an association is made between the users and a group identifier.

Presentation of a user interface for confirming unreliable group data

This disclosure describes a system for automatically identifying an item from among a variation of items of a same type. For example, an image may be processed and resulting item image information compared with stored item image information to determine a type of item represented in the image. If the matching stored item image information is part of a cluster, the item image information may then be compared with distinctive features associated with stored item image information of the cluster to determine the variation of the item represented in the received image.

Item identification among a variant of items

Determining an item involved in an event at an event location

In one or more implementations, groups of users enter a facility within a short time. Automated systems attempt to predict a grouping for various users, but the predicted grouping may not exhibit a confidence level above a certain threshold level. In such cases, a user interface is generated and presented to an associate to confirm the grouping of users. The user interface includes an image gallery and interface elements that allow for the selection of users into a group. The image gallery includes previously stored images of users, such as images obtained during a registration process or during a previous visit to the facility. In addition, the user interface also includes image data obtained from a camera at the facility depicting visitors at various times or locations. Once a grouping has been selected or confirmed by an associate, group data may be generated and stored for the group of users.

Generation of image gallery for acquisition of group data

This disclosure describes a system for assisting users in locating items from inventory locations and/or obtaining additional information about items. In some instances, a user may experience difficulty in locating an inventory item in a materials handling facility. By detecting one or more expressions presented by the user, an inventory management system may determine that assistance is to be provided to the user, determine and provide the appropriate assistance to the user.

Expression and gesture based assistance

Growing community of inventors

Snoqualmie, WA, United States of America

Sudarshan Narasimha Raghavan