Amazon Technologies, Inc.

Connor Spencer Blue Worley

Tomer Daniel Moscovich

Nishitkumar Ashokkumar Desai

Jayakrishnan Kumar Eledath

Petko Tsonev

Claire Law

Behjat Siddiquie

Devin Bertrum Pauley

Jue Wang

Julien George Beguin

Hue Tuan Thi

Bharat Singh

Gerard Guy Medioni

John Aaron Zarraga

Ohil Krishnamurthy Manyam

Gopi Prashanth Gopal

Navid Shiee

Jean Laurent Guigues

Oliver Huy Doan

Kun Qian

Chuhang Zou

Paul Ignatius Dizon Echevarria

Matthew Fletcher Harrison

Jung Sik Yang

Thomas Bruno Mader

Edward Albert Liljegren

Pahal Kamlesh Dalal

Peter A Larsen

This disclosure describes techniques for updating planogram data associated with a facility. The planogram may indicate inventory locations within the facility for various types of items supported by product fixtures. In particular an image of a product fixture is analyzed to identify image segments corresponding to product groups, where each product group consists of instances of the same product and each image segment corresponds to a group of image points. Image data is further analyzed to determine coordinates of the points of each image segment. A product space corresponding to the product group is then defined based on the coordinates of the points of the product group. In some cases, for example, a product space may be defined in terms of the coordinates of the corners of a rectangular bounding box or volume.

Image-based detection of planogram product spaces

This disclosure describes techniques for updating planogram data associated with a facility. The planogram may indicate, for different shelves and other inventory locations within the facility, which items are on which shelves. For example, the planogram data may indicate that a particular item is located on a particular shelf. Therefore, when a system identifies that a user has taken an item from that shelf, the system may update a virtual cart of that user to indicate addition of the particular item. In some instances, however, a new item may be stocked on the example shelf instead of a previous item. The techniques described herein may use sensor data generated in the facility to identify this change and update the planogram data to indicate an association between the shelf and the new item.

Inferring facility planograms

Described are systems for processing sensor data associated with an inventory location for the purpose of identifying stable and unstable states in the sensor data. The stable and unstable states can be determined by fitting a line or curve to a portion of the sensor data occurring during a window of time and calculating a corresponding slope value of the line or curve. By comparing the slope value of the line or curve to a threshold value, stable and unstable states can be determined for the sensor data. Once stable states have been identified, a change in sensor data can be determined and used to identify a change in quantity of items at the inventory location.

Sensor data processing for detection of changes in item quantities

Described are systems for processing sensor data obtained from a sensor such as a weight sensor. In one implementation, a line may be fit to a set of sensor data occurring during a window of time. A slope value of the line may be calculated and provided as input to a change detector. The change detector may implement a cumulative sum (CUSUM) algorithm to determine when the slope value is indicative of a stable or unstable state. Values of sensor data associated with times indicated as having a stable state may be used to determine weight data.

Sensor data processing system

Techniques for determining whether touch-input gestures approximate straight lines and for animating a display with such gestures are described. The techniques determine a linear regression line for pixel locations comprising a gesture, determine distances of the pixel locations from the linear regression line, and render the set of pixel locations to the display based on the distances and a threshold.

Straight line gesture recognition and rendering

Techniques are described for generating a composite image that depicts a first object with a second object. Two-dimensional images of the first object are captured, along with depth information that includes three-dimensional coordinates of points on the surface of the first object. Based on the depth information, a polygonal model may be determined for the first object including color information determined from the images. The polygonal model of the first object may be placed with a polygonal model of the second object in a virtual scene, and ray tracing operations may generate a plurality of pixels for the composite image. In cases where the first object represents a user and the second object represents a product, the composite image may provide a substantially realistic preview of the user with the product.

Augmented reality images based on color and depth information

Control of head-mounted display (HMD) systems is described. One method displays at least one image on a first display layer of a HMD system. While displaying the at least one image, the method adjust a transparency setting of a second display layer from a first value to a second value to cause the second display layer to be opaque to limit an amount of light that passes through the lens for viewing the at least one image on the first display layer.

Controlling display layers of a head-mounted display (HMD) system

An augmented experience improves user experience by including virtual reflections of an actual background on virtual items presented in a user interface. An augmented image comprising a representation of a virtual item with one or more reflective surfaces is generated and presented in a user interface. Virtual reflections based on images of an actual background acquired by a camera are generated. The virtual reflections are superimposed on the one or more reflective surfaces of the virtual item for presentation of the augmented image. During presentation of the virtual item, the inclusion of the virtual reflection may improve overall realism of the virtual item.

Augmented reality presentation

Described are methods and systems of processing three-dimensional ('3D') data by generating an edge map, a depth map, or both. Data points are placed into a bin array based their respective image coordinates. The data points in each bin are processed to determine edge data. An edge map may be generated from this edge data. A bin value may be generated based on the data points in each bin, and a depth map generated using these bin values. The edge data and the edge map may be processed using one or more filter functions. Measurements based on the edge map may be provided at a resolution greater than that available with the depth map.

Processing unordered point cloud

Techniques for determining whether touch-input gestures approximate straight lines and for animating a display with such gestures are described. Some techniques receive a gesture input through a display, define an oriented bounding box for the gesture, and determine whether the gesture approximates a straight line based on an amount of the gesture that is contained within the oriented bounding box. Other techniques determine a linear regression line for pixel locations comprising a gesture, determine distances of the pixel locations from the linear regression line, and render the set of pixel locations to the display based on the distances and a threshold. Further techniques record a gesture input through a display, determine a sequence of frame updates for the gesture, each frame update including a part of the gesture, and provide the frame updates for sequential updating of the display to effect an animation of the gesture on the display.

Described are methods and systems of providing an augmented experience on a user device to facilitate user interaction with one or more virtual items. An augmented image comprising an actual object and a virtual item is generated and presented in a user interface. The user interface allows the user to lock a relative position of the virtual item as presented, such that the user may appear to 'move' the virtual item. The user interface may also provide sizing information of the virtual item relative item to the actual object.

Single-touch immersion control of head-mounted display (HMD) systems is described. One method outputs video from an electronic device to a HMD system that includes a display layer and a variable-transparency layer. The electronic device controls the variable-transparency layer to operate in a first state in which the variable-transparency layer is transparent and to operate in a second state in which the variable-transparency layer is opaque. The electronic device switches between the second state and the first state in response to a single-touch event detected by the electronic device.

Growing community of inventors

San Diego, CA, United States of America

Connor Spencer Blue Worley