Amazon Technologies, Inc.

Waqas Syed Ahmed

Danny Guan

Russell Dicker

David Echevarria Ignacio

Prathiban Mohanasundaram

Nishitkumar Ashokkumar Desai

Helene Causse

Joseph Spencer King

Brett Richard Taylor

Mirko Mandic

Scott Allen Mongrain

Florin V Manolache

Shannon Thomas

Sabarishkumar Iyer

Sudarshan Narasimha Raghavan

Sridhar Boyapati

Jason Michael Famularo

Thomas Meilandt Mathiesen

Maren Marie Costa

Casey Louis Thurston

Jacques Daniel Thomas

Aaron D Wilson

Christopher Richard Fescoe

Val Dan Dar Ion I Rosca

Corneliu Gabriel Alexandru Rudeanu

Fei Xiong

Caylan Lee

Joseph M Riley

Dilip Kumar

Jiajun Zhao

Kartik Muktinutalapati

Joseph M Alyea

Valentin Radu Munteanu

Longlong Zhu

Ivan Stankovic

Adrian Quark

Leo Parker Dirac

Kevin Michael Small

Aditya Rekha Dharanipragada

Alexander J Cox

Amber Kathleen Joehnk

Hannes Scheidler

Adrian Bell Barkus

Peiran Zhou

Liem Dinh

Ao Yan

Saranya Karuppusamy

Guy Levanon

Simon Hachey

Product spaces for containers of non-discretized items are defined by sets of points in three-dimensional space. When a customer breaches or contacts a product space established for a container of the non-discretized items, video feeds representing the product space are displayed to an associate in a web page or another user interface. By interacting with the user interface, the associate may view the video feeds and make a determination as to whether the customer received any of the non-discretized items from the container, and to confirm an identity of the customer accordingly.

Detecting and confirming interactions by proximity and video

This disclosure is directed to techniques in which a first user in an environment scans visual indicia associated with an item, such as a barcode, before handing the item to a second user. One or more computing devices may receive an indication of the scan, retrieve image data of the interaction from a camera within the environment, identify the user that received the item, and update a virtual cart associated with the second user to indicate addition of the item.

Analyzing sensor data to identify events

Described are systems and methods for detecting objects using calibrated imaging devices and obfuscating, in real-time or near real time, portions of the video data to protect the privacy of operators represented in the video data. For example, a position of an operator within a fulfillment center may be determined or tracked in video data and the pixels representative of that operator may be obfuscated using pixilation and/or other techniques so that a reviewing agent that is viewing the video data cannot determine the identity of the operator. Such obfuscation may be performed in real-time or near real-time using automated processing. In addition, only portions of the video data may be obfuscated so that events (e.g., item picks, item place) and/or other objects represented in the video data are still viewable to the reviewing agent.

Obfuscating portions of video data

Techniques for employing user interfaces to output information indicative of events occurring in an inventory facility, and receive feedback from a human regarding the events are described herein. In one implementation, an event may take place in an inventory facility, such as a customer taking an item from an inventory location, returning an item to an inventory location, and so forth. An automated system of an inventory management system may process sensor data collected by sensors in the inventory facility to determine details of the event. In some examples, the inventory management system is unable to determine with a high level of confidence what occurred during the event. The inventory management system may provide the sensor data to a human associate through an associate interface, and receive input regarding details of the event from the human associate through the associate interface.

Identifying user-item interactions in an automated facility

A system may generate image data of users within a facility. Such users may be part of a group that is unknown to the system. The system can predict group data for two or more users based on a resemblance or the users being within a threshold distance of each other. However, if a confidence level associated with the predicted group data is below a threshold value, the group data is deemed unreliable and assistance from an associate is deemed necessary. A user interface that includes a portion of the image data, information about the predicted group data, and other interface elements is presented to the associate via a display. Based on the input data received from the associate, the group data can be confirmed or rejected. If the group data is confirmed, an association is made between the users and a group identifier.

Presentation of a user interface for confirming unreliable group data

This disclosure describes interactive data processing systems configured to facilitate selection by a human associate of tentative results generated by an automated system from sensor data. In one implementation, an event may take place in a materials handling facility. The event may comprise a pick or place of an item from an inventory location, movement of a user, and so forth. The sensor data associated with the event is processed by an automated system to determine tentative results associated with the event. In some situations, an uncertainty may exist as to which of the tentative results accurately reflects the actual event. The system may then determine whether the event is to be merged with one or more temporally and spatially proximate events and, if so, the sensor data and tentative results for the merged event is sent to a human associate. The associate may select one of the tentative results.

Merging events in interactive data processing systems

In one or more implementations, groups of users enter a facility within a short time. Automated systems attempt to predict a grouping for various users, but the predicted grouping may not exhibit a confidence level above a certain threshold level. In such cases, a user interface is generated and presented to an associate to confirm the grouping of users. The user interface includes an image gallery and interface elements that allow for the selection of users into a group. The image gallery includes previously stored images of users, such as images obtained during a registration process or during a previous visit to the facility. In addition, the user interface also includes image data obtained from a camera at the facility depicting visitors at various times or locations. Once a grouping has been selected or confirmed by an associate, group data may be generated and stored for the group of users.

Generation of image gallery for acquisition of group data

Users may enter a facility singly or in groups. A user interface may be presented to an associate that includes information obtained from sensors in the facility. The associate may utilize the information presented to determine if a user is alone, or to associate that user with a particular group of users. Input from the associate obtained from this user interface may then be used by a facility management system during operation to attribute the actions of the users to a particular account.

User interface for acquisition of group data

Growing community of inventors

Bellevue, WA, United States of America

Waqas Syed Ahmed