Amazon Technologies, Inc.

Danny Guan

Waqas Syed Ahmed

David Echevarria Ignacio

Prathiban Mohanasundaram

Nishitkumar Ashokkumar Desai

Helene Causse

Pragyana K Mishra

Brett Richard Taylor

Mirko Mandic

Shannon Thomas

Sabarishkumar Iyer

Sudarshan Narasimha Raghavan

Sridhar Boyapati

Jules Cook Graybill

Jason Michael Famularo

Thomas Meilandt Mathiesen

Maren Marie Costa

Casey Louis Thurston

Jacques Daniel Thomas

Christopher Richard Fescoe

Fei Xiong

Caylan Lee

Joseph M Riley

Dilip Kumar

James Christopher Curlander

Jiajun Zhao

Jonathan Robert Dughi

Kartik Muktinutalapati

Joseph M Alyea

Longlong Zhu

Adrian Quark

David Joseph Rendleman

Ivan Stankovic

Adrian Bell Barkus

Peiran Zhou

Liem Dinh

Ao Yan

Saranya Karuppusamy

Product spaces for containers of non-discretized items are defined by sets of points in three-dimensional space. When a customer breaches or contacts a product space established for a container of the non-discretized items, video feeds representing the product space are displayed to an associate in a web page or another user interface. By interacting with the user interface, the associate may view the video feeds and make a determination as to whether the customer received any of the non-discretized items from the container, and to confirm an identity of the customer accordingly.

Detecting and confirming interactions by proximity and video

This disclosure is directed to techniques in which a first user in an environment scans visual indicia associated with an item, such as a barcode, before handing the item to a second user. One or more computing devices may receive an indication of the scan, retrieve image data of the interaction from a camera within the environment, identify the user that received the item, and update a virtual cart associated with the second user to indicate addition of the item.

Analyzing sensor data to identify events

Described are systems and methods for detecting objects using calibrated imaging devices and obfuscating, in real-time or near real time, portions of the video data to protect the privacy of operators represented in the video data. For example, a position of an operator within a fulfillment center may be determined or tracked in video data and the pixels representative of that operator may be obfuscated using pixilation and/or other techniques so that a reviewing agent that is viewing the video data cannot determine the identity of the operator. Such obfuscation may be performed in real-time or near real-time using automated processing. In addition, only portions of the video data may be obfuscated so that events (e.g., item picks, item place) and/or other objects represented in the video data are still viewable to the reviewing agent.

Obfuscating portions of video data

Techniques for employing user interfaces to output information indicative of events occurring in an inventory facility, and receive feedback from a human regarding the events are described herein. In one implementation, an event may take place in an inventory facility, such as a customer taking an item from an inventory location, returning an item to an inventory location, and so forth. An automated system of an inventory management system may process sensor data collected by sensors in the inventory facility to determine details of the event. In some examples, the inventory management system is unable to determine with a high level of confidence what occurred during the event. The inventory management system may provide the sensor data to a human associate through an associate interface, and receive input regarding details of the event from the human associate through the associate interface.

Identifying user-item interactions in an automated facility

A system may generate image data of users within a facility. Such users may be part of a group that is unknown to the system. The system can predict group data for two or more users based on a resemblance or the users being within a threshold distance of each other. However, if a confidence level associated with the predicted group data is below a threshold value, the group data is deemed unreliable and assistance from an associate is deemed necessary. A user interface that includes a portion of the image data, information about the predicted group data, and other interface elements is presented to the associate via a display. Based on the input data received from the associate, the group data can be confirmed or rejected. If the group data is confirmed, an association is made between the users and a group identifier.

Presentation of a user interface for confirming unreliable group data

This disclosure describes interactive data processing systems configured to facilitate selection by a human associate of tentative results generated by an automated system from sensor data. In one implementation, an event may take place in a materials handling facility. The event may comprise a pick or place of an item from an inventory location, movement of a user, and so forth. The sensor data associated with the event is processed by an automated system to determine tentative results associated with the event. In some situations, an uncertainty may exist as to which of the tentative results accurately reflects the actual event. The system may then determine whether the event is to be merged with one or more temporally and spatially proximate events and, if so, the sensor data and tentative results for the merged event is sent to a human associate. The associate may select one of the tentative results.

Merging events in interactive data processing systems

In one or more implementations, groups of users enter a facility within a short time. Automated systems attempt to predict a grouping for various users, but the predicted grouping may not exhibit a confidence level above a certain threshold level. In such cases, a user interface is generated and presented to an associate to confirm the grouping of users. The user interface includes an image gallery and interface elements that allow for the selection of users into a group. The image gallery includes previously stored images of users, such as images obtained during a registration process or during a previous visit to the facility. In addition, the user interface also includes image data obtained from a camera at the facility depicting visitors at various times or locations. Once a grouping has been selected or confirmed by an associate, group data may be generated and stored for the group of users.

Generation of image gallery for acquisition of group data

Diagnostics may be performed on imaging devices such as digital cameras that are provided in large numbers, or mounted in arrays or networks, by providing imaging data captured from such devices to a machine learning system or classifier that has been trained to recognize anomalies based on imaging data. The machine learning system or classifier may be trained using a training set of imaging data previously captured by one or more imaging devices that has been labeled with regard to whether such imaging devices encountered any anomalies when the imaging data was captured, and if so, which anomalies were encountered. Additionally, a perceptual score which represents the quality of a given image or imaging data may be calculated and used to rank or define the image or imaging data in terms of quality, or determine whether the image or imaging data is suitable for its intended purpose.

Automatic large-scale imaging device diagnostics

Growing community of inventors

Seattle, WA, United States of America

Danny Guan