7-eleven, Inc.

Sarath Vakacharla

Shahmeer Ali Mirza

Sailesh Bharathwaaj Krishnamurthy

Trong Nghia Nguyen

Deepanjan Paul

Caleb Austin Boulio

Madan Mohan Chinnam

Matthew Raymond Magee

Crystal Maung

Maninder Singh Suri

Ravi Kumar Kurva

Fahad Mirza

Ranganathan Mohan

Jonathan Christopher Hodge

Joshua E Berry

Franklin Philips

Ryan Robert Strong

Trang Nghia Nguyen

An object tracking system that includes a sensor and a tracking system. The sensor is configured to capture a frame of at least a portion of a rack within a global plane for a space. The tracking system is further configured to receive the frame, to determine a pixel location for the first person, and to determine a person is within the predefined zone associated with the rack. The tracking system is further configured to identify the plurality of items in a digital cart associated with the person, to identify an item from the digital cart associated with the first person, and to remove the identified item from the digital cart associated with the person.

Detecting and identifying misplaced items using a sensor array

A system includes a first sensor configured to generate images of at least a first portion of a space. A processor of the system is configured to determine a position of a possible object in the space based on generated images.

System and method for object tracking

An apparatus includes a display, interface, and processor. The interface receives video from a camera located in a physical store and directed at a first physical rack. The camera captures video of the rack during a shopping session. The processor displays a first virtual rack that emulates the first physical rack and includes first and second virtual shelves. The virtual shelves include virtual items, which include graphical representations of physical items located on the physical rack. The processor displays the rack video, which depicts an event including the person interacting with the first physical rack. The processor also displays a virtual shopping cart. The processor receives information associated with the event, identifying the first virtual item. The rack video depicts that the person selected the first physical item while interacting with the first physical rack. The processor then stores the first virtual item in the virtual shopping cart.

System and method for populating a virtual shopping cart based on video of a customer's shopping session at a physical store

An apparatus includes a processor. The processor receives an algorithmic shopping cart that includes a first set of items determined by an algorithm to have been selected by a person during a shopping session in a physical store, based on a set of inputs received from sensors located within the physical store. The processor also receives a virtual shopping cart that includes a second set of items. Video of the shopping session was captured by a set of cameras located in the physical store and depicts the person selecting the second set of items. The processor compares the algorithmic cart to the virtual cart and determines that a discrepancy exists between the algorithmic cart and the virtual cart. The processor determines a subset of the set of inputs associated with the discrepancy and attaches metadata explaining the discrepancy to the subset. The processor uses the subset to train the algorithm.

Feedback and training for a machine learning algorithm configured to determine customer purchases during a shopping session at a physical store

A system includes a sensor, a weight sensor, and a tracking subsystem. The tracking subsystem receives an image feed of top-view images generated by the sensor and weight measurements from the weight sensor. The tracking subsystem detects an event associated with an item being removed from a rack in which the weight sensor is installed. The tracking subsystem determines that a first person or a second person may be associated with the event. In response to determining that the first or second person may be associated with the event, buffer frames are stored of top-view images generated by the sensor during a time period associated with the event. The tracking subsystem then determines, using at least one of the stored buffer frames and a first action-detection algorithm, whether an action associated with the event was performed by the first person or the second person.

Action detection during image tracking

A system includes a sensor and a tracking subsystem. The tracking subsystem receives an image feed of top-view images generated by the sensor. The tracking subsystem detects an event associated with an item being removed from a rack. The tracking subsystem determines that a first person and a second person may be associated with the event. In response, the tracking subsystem dilates contours associated with the first and second person from a first depth to a second depth until the contours enter a zone adjacent to the rack. A number of iterations is determined for each contour to enter the zone adjacent to the rack. If the first person's contour enters the zone in fewer iterations, the item is assigned to the first person.

Image-based action detection using contour dilation

A scalable tracking system includes a camera subsystem, a weight subsystem, and a central server. The camera subsystem includes cameras that capture video of a space, camera clients that determine local coordinates of objects in the captured videos, and a camera server that determines the physical positions of objects in the space based on the determined local coordinates. The weight subsystem determines when items were removed from shelves. The central server determines which object in the space removed the items based on the physical positions of the objects in the space and the determination of when items were removed.

Scalable position tracking system for tracking position in large spaces

An object tracking system that includes a sensor and a tracking system. The sensor configured to capture a frame of at least a portion of a rack within a global plane for a space. The tracking system is configured to detect an item was removed from the rack. The tracking system is further configured to receive the frame of the rack, to identify a marker on an item within a predefined zone in the frame, and to identify the item associated with the identified marker. The tracking system is further configured to determine a pixel location for a person, to determine the person is within the predefined zone associated with the, and to add the identified item to a digital cart associated with the person.

Identifying non-uniform weight objects using a sensor array

An object tracking system includes a first sensor, a second sensor, and a tracking system. The tracking system is configured to determine that the first current pixel location for the shelf marker does not match a first expected pixel location for the shelf marker. The tracking system is further configured to determine a second current pixel location for the shelf marker within a second frame from the second sensor, to recalibrate the first sensor when the second current pixel location for the shelf marker matches the second pixel location for the shelf marker and to update the first pixel location with the first current pixel location and the second pixel location with the second current pixel location when the second current pixel location for the shelf marker does not match the second pixel location for the shelf marker.

Shelf position calibration in a global coordinate system using a sensor array

An apparatus includes a display, interface, and processor. The interface receives a live camera feed from a camera directed at a first physical structure located in a physical space. The processor receives an indication of an event associated with the first physical structure and accordingly displays a first virtual structure corresponding to the first physical structure. The first virtual structure includes virtual items that emulate physical items located on the first physical structure. The processor additionally displays a recording of the live camera feed, which depicts the event associated with the first physical structure.

System and method for presenting a virtual store shelf that emulates a physical store shelf

An object tracking system includes a sensor, a weight sensor, and a tracking system. The sensor is configured to capture a frame of at least a portion of a rack within a global plane for a space. The tracking system is configured to detect a weight decrease on the weight sensor. The tracking system is further configured to receive the frame of the rack, to determine a pixel location for a person, to determine the person is within a predefined zone associated with the rack. The tracking system is further configured to identify the item associated with the weight sensor and to add the identified item to a digital cart associated with the person.

Topview item tracking using a sensor array

An object tracking system includes a sensor and a tracking system. The sensor is configured to capture a frame of at least a portion of a physical space within a global plane for a space. The tracking system is configured to receive the frame, to detect an object within a zone of the frame, and to determine a pixel location for the object. The tracking system is further configured to identify a zone of the physical structure based on the pixel location, to identify an item based on the identified zone.

Detecting shelf interactions using a sensor array

A system includes a sensor and a tracking subsystem. The tracking subsystem receives an image feed of top-view images generated by the sensor and detects an event associated with a portion of a person entering a zone adjacent to a rack. The tracking subsystem determines that a first and second person may be associated with the event. The subsystem tracks the item and calculates a velocity of the item as it is moved through the space. The subsystem identifies, based on the calculated velocity, a frame in which the velocity of the item is less than a threshold velocity. The subsystem determines whether the first or second person is nearer the item in the identified frame. If the first person is nearer, the item is assigned to the first person.

Object assignment during image tracking

A scalable tracking system processes video of a space to track the positions of objects within a space. The tracking system determines local coordinates for the objects within frames of the video and then assigns these coordinates to time windows based on when the frames were received. The tracking system then combines or clusters certain local coordinates that have been assigned to the same time window to determine a combined coordinate for an object during that time window.

Growing community of inventors

Irving, TX, United States of America

Sarath Vakacharla