7-eleven, Inc.

Crystal Maung

Sailesh Bharathwaaj Krishnamurthy

Shahmeer Ali Mirza

Sumedh Vilas Datar

Tejas Pradip Rode

Shantanu Yadunath Thakurdesai

Sarath Vakacharla

Madan Mohan Chinnam

Deepanjan Paul

Matthew Raymond Magee

Kyle Dalal

Xinan Wang

Trong Nghia Nguyen

Trang Nghia Nguyen

A system for capturing images for training an item identification model obtains an identifier of an item. The system detects a triggering event at a platform, where the triggering event corresponds to a user placing the item on a platform. At least one camera captures an image of the item. The system extracts a set of features associated with the item from the image. The system associates the item to the identifier and the set of features. The system adds a new entry to a training dataset of the item identification model, where the new entry represents the item labeled with the identifier and the set of features.

System and method for capturing images for training of an item identification model

A device configured to detect a triggering event at a platform and to capture a depth image of items on the platform using a three-dimensional (3D) sensor. The device is further configured to determine an object pose for each item on the platform and to identify one or more cameras from among a plurality of cameras based on the object pose for each item on the platform. The device is further configured to capture one or more images of the items on the platform using the identified cameras and to identify items within the one or more images based on features of the items. The device is further configured to identify a user associated with the identified items on the platform, to identify an account that is associated with the user, and to associate the identified items with the account of the user.

Item identification using digital image processing

A device configured to receive a first point cloud data for a first item, to identify a first plurality of data points for the first object within the first point cloud data, and to extract the first plurality of data points from the first point cloud data. The device is further configured to receive a second point cloud data for the first item, to identify a second plurality of data points for the first object within the second point cloud data, and to extract a second plurality of data points from the second point cloud data. The device is further configured to merge the first plurality of data points and the second plurality of data points to generate combined point cloud data and to determine dimensions for the first object based on the combined point cloud data.

Determining dimensions of an item using point cloud information

A device configured to capture a first image of an item on a platform using a camera and to determine a first number of pixels in the first image that corresponds with the item. The device is further configured to capture a first depth image of an item on the platform using a three-dimensional (3D) sensor and to determine a second number of pixels within the first depth image that corresponds with the item. The device is further configured to determine that the difference between the first number of pixels in the first image and the second number of pixels in the first depth image is less than the difference threshold value, to extract the plurality of pixels corresponding with the item in the first image from the first image to generate a second image, and to output the second image.

Image cropping using depth information

A device configured to identify a first pixel location within a first plurality of pixels corresponding with an item in a first image and to apply a first homography to the first pixel location to determine a first (x,y) coordinate. The device is further configured to identify a second pixel location within a second plurality of pixels corresponding with the item in a second image and to apply a second homography to the second pixel location to determine a second (x,y) coordinate. The device is further configured to determine that the distance between the first (x,y) coordinate and the second (x,y) coordinate is less than or equal to the distance threshold value, to associate the first plurality of pixels and the second plurality of pixels with a cluster for the item, and to output the first plurality of pixels and the second plurality of pixels.

Item location detection using homographies

A device configured to detect a triggering event corresponding with a user placing a first item on the platform, to capture a first image of the first item on the platform using a camera, and to input the first image into a machine learning model that is configured to output a first encoded vector based on features of the first item that are present in the first image. The device is further configured to identify a second encoded vector in an encoded vector library that most closely matches the first encoded vector and to identify a first item identifier in the encoded vector library that is associated with the second encoded vector. The device is further configured to identify the user, to identify an account that is associated with the user, and to associate the first item identifier with the account of the user.

Item identification using multiple cameras

A device configured to receive a first encoded vector and receive one or more feature descriptors for a first object. The device is further configured to remove one or more encoded vectors from an encoded vector library that are not associated with the one or more feature descriptors and to identify a second encoded vector in the encoded vector library that most closely matches the first encoded vector based on the numerical values within the first encoded vector. The device is further configured to identify a first item identifier in the encoded vector library that is associated with the second encoded vector and to output the first item identifier.

Reducing a search space for item identification using machine learning

A system for capturing images for training an item identification model obtains an identifier of an item. The system detects a triggering event at a platform, where the triggering event corresponds to a user placing the item on a platform. The system causes the platform to rotate. The system causes at least one camera to capture an image of the item while the platform is rotating. The system extracts a set of features associated with the item from the image. The system associates the item to the identifier and the set of features. The system adds a new entry to a training dataset of the item identification model, where the new entry represents the item labeled with the identifier and the set of features.

A system for identifying items based on aggregated metadata obtains images of an item. The system extracts a set of features from images of the item. The system identifies a first value of a first feature associated with a first image of the item. The system identifies a second value of the first feature associated with a second image of the item. The system aggregates the first value and the second value. The system associates the item to the aggregated first value and the second value, where the aggregated first value and the second value represent the first feature of the item. The system adds a new entry for each image of the item to a training dataset associated with an item identification model.

System and method for aggregating metadata for item identification using digital image processing

A system includes sensors and a processor that tracks first and second objects in a space. Upon detecting that a tracked position of the first object is within a threshold distance of a tracked position of a second object, a top-view image of the first object is received from a first sensor. Based on this top-view image, a first descriptor is determined for the first object. The first descriptor is associated with an observable characteristic of the first object. The processor identifies the first object based at least in part upon the first descriptor.

Object re-identification during image tracking

An object tracking system includes a sensor and a tracking system. The sensor is configured to capture a first frame of a global plane for at least a portion of a space. The tracking system is configured to receive a first coordinate in the global plane where a first marker is located in the space and to receive a second coordinate in the global plane where a second marker is located in the space. The tracking system is further configured to identify the first marker and the second marker within the first frame, to determine a first pixel location in the first frame for the first marker, to determine a second pixel location in the first frame for the second marker, and to generate a homography based on the first coordinate, the second coordinate, the first pixel location, and the second pixel location.

Growing community of inventors

Dallas, TX, United States of America

Crystal Maung