Amazon Technologies, Inc.

Frank Florian Liberato, Jr

Daniel Bibireata

Yasser Baseer Asmi

Nishitkumar Ashokkumar Desai

Ammar Chinoy

Muralidhar Koka

Joachim Sebastian Stahl

Shrivatsan Vasudhevan

Pranab Mohanty

Sudarshan Narasimha Raghavan

Emilio Ian Maldonado

Jiajun Zhao

Jaechul Kim

Maxwell Aaron Corman

I-Hung Wang

Xiaofeng Ren

Prafulla Jinendra Masalkar

David Allen Smith

Hao Jiang

Chen Xu

Todd Nelson Schoepflin

Bradley David Volen

Described is a multiple-camera system and process for re-identifying a user located in a materials handling facility based on user patterns and/or descriptors representative of the user. In one implementation, a user pattern and/or a plurality of descriptors representative of a user are maintained as a position of a user is tracked through a materials handling facility. If the tracking of the user is lost, the last known position is stored with the user pattern and/or descriptors. If a new object is detected and confirmed to be a user, a user pattern and/or descriptors of the new object are compared with the stored user pattern and/or descriptors to determine if the new object is the user.

Updating a position of a user within a materials handling facility

Described are systems and techniques to generate data for display to present visualizations of data acquired from sensors in a facility. The data visualizations may be used to develop, configure, administer, or otherwise support operation of the facility. In one implementation, the visualization may include a view incorporating aggregated images acquired from multiple cameras, depth data, tracking information about objects in the facility, and so forth. An analyst may use the data visualization to determine occurrence of an action in the facility such as a pick of an item, place of an item, what item was involved with an action, what user was involved with the action, and so forth. Based on the information presented by the data visualization, changes may be made to data processing parameters.

Visualization system for sensor data and facility data

Described is a multiple-camera system and process for detecting a user within a materials handling facility and tracking a position of the user as the user moves through the materials handling facility. In one implementation, a plurality of depth sensing cameras are positioned above a surface of the materials handling facility and oriented to obtain a top-down view of the surface of the materials handling facility, along with any objects (e.g., users) on the surface of the materials handling facility. The depth information from the cameras may be utilized to detect objects on the surface of the materials handling facility, track a movement of those objects and determine if those objects are users.

Collecting user pattern descriptors for use in tracking a movement of a user within a materials handling facility

Described is a multiple-camera system for use in capturing images of users within a materials handling facility and processing those images to monitor the movement of users. For large materials handling facilities, a large number of cameras may be required to monitor the facility. Processing of the data generated from a large number of cameras becomes difficult. The implementations described herein include a hierarchy that allows image data from any number of cameras within a materials handling facility to be processed without substantially increasing the processing time needed or sacrificing processing capabilities.

Monitoring locations within a materials handling facility

Re-establishing tracking of a user within a materials handling facility

Described is a multiple-camera system and process for detecting a user within a materials handling facility and tracking a position of the user as the user moves through the materials handling facility. In one implementation, a plurality of depth sensing cameras are positioned above a surface of the materials handling facility and oriented to obtain an overhead view of the surface of the materials handling facility, along with any objects (e.g., users) on the surface of the materials handling facility. The depth information from the cameras may be utilized to detect objects on the surface of the materials handling facility, track a movement of those objects and determine if those objects are users.

Using depth sensing cameras positioned overhead to detect and track a movement of a user within a materials handling facility

Frames of sensor data may be obtained from many sensors arranged throughout a facility. These frames may be time synchronized to support further processing. For example, frames containing image data obtained at about the same time from many cameras within the facility may be used to create an aggregate or 'stitched' view of the facility at that time. The synchronization may involve storing the frames from several sensors in buffers. A time window may be specified and used in conjunction with timestamps of the frames to select a set of sensor data from the buffers that are deemed to be synchronized data. The synchronized data may then be used for further processing.

Data synchronization system

Camera hierarchy for monitoring large facilities

Data visualization system

Processing image data from a camera cluster

Systems involving a plurality of cameras with clocks may not remain time synchronized during operation. Described in this disclosure are techniques for synchronizing one or more of the clocks of a plurality of cameras or the images produced by the plurality of cameras. In one implementation, a timestamp projector produces an optical timestamp encoding data indicative of timing. One or more cameras may acquire images of a scene that include the optical timestamp. The images may be processed to recover the data indicative of timing. This data may be used to set the clock of the camera, set timestamps associated with the images for subsequent use, and so forth.

Camera time synchronization system

Growing community of inventors

Kirkland, WA, United States of America

Frank Florian Liberato, Jr