Amazon Technologies, Inc.

Jared Joseph Frank

Jason Michael Famularo

Gianna Lise Puerini

Dilip Kumar

Amber Autrey Taylor

Thomas Meilandt Mathiesen

Korwin Jon Smith

Benjamin Ralph Hollis

Sridhar Boyapati

Alexander Michael McNamara

Ammar Chinoy

Pranab Mohanty

Joachim Sebastian Stahl

Wyatt David Camp

Described is a system and method for proactively determining and resolving events having low confidence scores and/or resolving disputed events. For example, when an event, such as a pick of an item from an inventory location within a materials handling facility occurs, the event aspects (e.g., user involved in the event, item involved in the event, action performed) are determined based on information provided from one or more input components (e.g., camera, weight sensor). If each event aspect cannot be determined with a high degree of confidence, the event information is provided to an associate for resolution.

Event image information retention

Described is a system and method for presenting event information to a user and, if necessary, obtaining confirmation of different aspects (user, item, action) of the event. In some implementations, an event includes a user, an action, and an item. For example, an event may include a user picking an item from an inventory location, a user placing an item into a tote associated with the user, etc. if the aspects of the event cannot be determined with a high enough degree of confidence, a user interface may be generated and sent to the user requesting confirmation of one or more of the aspects of the event.

Item and action determination

Described are systems and techniques to present information to a user of a materials handling facility using a representative color. A representative color at an inventory location is determined based on one or more colors used at the inventory location. Once determined, the representative color may be used in the presentation of a user interface that includes item data and visual elements therein. The use of the representative color in the user interface facilitates the identification of items or an inventory location within the material handling facility.

User interface with a representative color

Markers may be presented on different objects in a facility, such as shelves or totes. A marker can use color, placement of elements, blending effects, or infrared light layers to encode data about the object. During operation, a camera within the facility may acquire an image, which includes an image of the marker as presented on the object. The marker can be analyzed to detect marker data encoded therein. By decoding the marker data, details about the marker can be extracted, such as an identification of the object. In addition, using details about the marker location within the image and location data associated with the camera, a location of the object relative to the camera can be determined, as well as an orientation of the object relative to the camera. As the object moves within the facility, the object can be tracked based on this location and orientation data.

Tracking of objects using markers

Proactive resolution of event information

Machine-readable optical markers may be presented on a surface of a shelf, on a display device, or on a structure. Markers are presented as an array of graphical elements arranged in rows and columns. The graphical elements in the array of the mark utilize color and placement to encode marker data. In addition, a source image graphic may also be blended with a portion of the graphical elements in the array to create a blended image. In other embodiments, the marker may be presented with one or more borders surrounding the perimeter of the graphical elements in the array. The marker data may be presented using infrared light with a visible light layer presented over the array of graphical elements.

Presentation of machine-readable optical markers

Described are systems and techniques for using machine-readable optical markers to calibrate a machine vision system. In one implementation, the machine vision system may generate calibration data associating spatial coordinates with image coordinates of the machine-readable marker in an image acquired by a camera. The machine-readable marker may comprise a blend of an optical code that encodes marker data and a human-readable source graphic image. In some implementations, the optical code of the markers may be detectable under infrared light while remaining invisible to the unaided eye.

Machine vision calibration system with marker

Described are systems and techniques configured to determine and present information to a user of a materials handling facility. As the user moves through the facility, information is dynamically presented to the user on one or more displays using a user interface. The user interface may be presented on the display as the user approaches, and presentation may cease as the user moves away. The amount of information, size of elements presented on the display, and so forth may be based on kinematic data about the user. The kinematic data may include position, speed, orientation, and so forth.

Dynamic inventory management user interface

Color adaptable inventory management user interface

Determine an item and user action in a materials handling facility

Described is a system and method for presenting item information to a user based on the activities of the user. In one example, when a user picks an item from an inventory location, the item is identified, and a representation of the item, and optionally additional item information, is presented to the user as a confirmation that the item was properly identified. If the user adds the item to a tote associated with the user, an item identifier associated with the item is added to an item identifier list associated with the user. Likewise, an item identifier list representation, which provides information about all items identified on the item identifier list may be presented to the user as confirmation that the correct item has been added to the user item identifier list associated with the user.

Growing community of inventors

Snoqualmie, WA, United States of America

Jared Joseph Frank