7-eleven, Inc.

Frito-lay North America, Inc.

Shahmeer Ali Mirza

Sailesh Bharathwaaj Krishnamurthy

Sarath Vakacharla

Trong Nghia Nguyen

Deepanjan Paul

Crystal Maung

Matthew Raymond Magee

Sumedh Vilas Datar

Kyle Dalal

Madan Mohan Chinnam

Wilfred Marcellien Bourg, Jr

Ou Bai

Enrique Michel-Sanchez

Matthew O'Daniel Redmond

Scott Fagan

Caleb Austin Boulio

Chen C Shao

Scott G Richardson

John Robert Keller

Ravi Kumar Kurva

Xinan Wang

Fahad Mirza

Tejas Pradip Rode

Ryan Robert Strong

Maninder Singh Suri

Shantanu Yadunath Thakurdesai

Ranganathan Mohan

Jonathan Christopher Hodge

Joshua E Berry

Ryan Jerman Jones

Franklin Philips

Jon Andrew Crain

Pirooz Najafi

Shahid Quraishi

Scotty G Richardson

Trang Nghia Nguyen

Wilfred Marcellien Bourg

Wilfred Bourg

A system includes a sensor, a weight sensor, and a tracking subsystem. The tracking subsystem receives an image feed of top-view images generated by the sensor and weight measurements from the weight sensor. The tracking subsystem detects an event associated with an item being removed from a rack in which the weight sensor is installed. The tracking subsystem determines that a first person or a second person may be associated with the event. In response to determining that the first or second person may be associated with the event, buffer frames are stored of top-view images generated by the sensor during a time period associated with the event. The tracking subsystem then determines, using at least one of the stored buffer frames and a first action-detection algorithm, whether an action associated with the event was performed by the first person or the second person.

Action detection during image tracking

A system includes one or more memory units and a processor. The processor is configured to receive, from a food temperature probe, a first temperature associated with a first food item. The processor is further configured to receive, from the food temperature probe, a second temperature associated with a second food item. The processor is further configured to receive, from the food temperature probe, a third temperature that was measured by the food temperature probe after measuring the first temperature but before measuring the second temperature, the third temperature associated with a cleaning of the food temperature probe. The processor is further configured to send an alert for display on a user device when the third temperature is greater than the cleaning threshold temperature.

Systems and methods for monitoring food temperatures

An apparatus includes a display and a processor. The processor displays a virtual shopping cart. The processor also receives information indicating that an algorithm determined that a physical item was selected by a person during a shopping session in a physical store, based on a set of inputs received from sensors located within the store. In response, the processor displays a virtual item, which includes a graphical representation of the physical item. The processor additionally displays a rack video captured during the shopping session by a rack camera located in the store. The rack camera is directed at a physical rack located in the store, which includes the physical item. In response to displaying the rack video, the processor receives information identifying the virtual item, where the rack video depicts that the person selected the physical item. The processor then stores the virtual item in the virtual shopping cart.

System and method for populating a virtual shopping cart based on a verification of algorithmic determinations of items selected during a shopping session in a physical store

A system for determining an ambient concentration of compositions for bathroom cleaning comprises a processor operable to receive a concentration measurement from a sensor over a network within a first period of time. The processor is operable to compare the received concentration measurement to a first threshold and to a second threshold greater than the first threshold. The processor is operable to instruct a memory communicatively coupled to the processor to store an indication that a bathroom was cleaned in response to a determination that the received concentration measurement is greater than the first threshold and less than the second threshold. The processor is operable to send an alert for display on a user device indicating either that the sensor has been tampered or that a spill event has occurred in response to a determination that the received concentration measurement is greater than the second threshold.

System and method for determining an ambient concentration of compositions for bathroom cleaning

An object tracking system includes a sensor and a controller. The sensor captures frames for a global plane for at least a portion of a space. The controller has a memory that stores a homography associated with the sensor and a processor. The processor receives a first frame from the sensor and identifies a pixel location within the first frame. The processor determines an estimated sensor location for the sensor by applying the homography to the pixel location, and determines a location difference between the estimated sensor location and an actual sensor location for the sensor. The processor then compares the location difference to a difference threshold level, and recomputes the homography in response to determining that the location difference exceeds the difference threshold level.

Homography error correction

An object tracking system that includes a first sensor and a second sensor that are each configured to capture frames of at least a portion of a global plane for a space. The system is configured to identify a first pixel location for a marker within a first frame and to determine an (x,y) coordinate for the marker using a first homography. The system is further configured to identify a second pixel location for the marker in the second sensor using a second homography, to identify a third pixel location using a disparity mapping, and to determine a distance difference between the second pixel location and the third pixel location. The system is further configured to compare the distance difference to a difference threshold level and to recompute the first homography and/or the second homography in response to determining that the distance difference exceeds the difference threshold level.

Homography error correction using a disparity mapping

A system includes a longitudinal rack storing a plurality of items, a shoe movably attached to the rack, a magnet coupled to the shoe and a longitudinal circuit board arranged along the length of the rack. The circuit board includes an array of sensors along the length of the rack. Each sensor generates a value depending on a position of the magnet in relation to the sensor. The circuit board further includes a memory storing values generated by the sensors, and a processor configured to determine a position of the shoe/magnet based on the values and determine a count of the items based on the position of the shoe/magnet.

System and method for determining an item count in a rack using magnetic sensors

A system includes a rack storing items, an interaction device that processes the items, and a master controller that is configured to receive a communication signal from the rack indicating that items have been removed from the rack and determine based on monitoring communication signals from the interaction device whether the removed items were processed by the interaction device within a pre-configured time period from the time the items were removed from the rack. In response to determining that the items were not processed by the interaction device within a pre-configured time period, the master controller transmits an alert message including a number of the items removed from the rack and an identification associated with the items.

System and method for determining product theft information using magnetic sensors

A system for updating a training dataset of an item identification model determines that an item is not included in a training dataset. In response to determining that the item is not included in the training dataset, the system obtains an identifier of the item. The system detects a triggering event at a platform, where the triggering event corresponds to a user placing the item on a platform. The system captures images of the item. The system extracts a set of features associated with the item from the images. The system associates the item to the identifier and the set of features. The system adds a new entry to the training dataset, where the new entry represents the item labeled with the identifier and the set of features.

Database management system and method for updating a training dataset of an item identification model

An image sensor is positioned such that a field-of-view of the image sensor encompasses at least a portion of a rack storing items. The image sensor generates angled-view images of an object. A pixel position of a body part of a person is determined in at least a subset of the received image frames, thereby determining a set of pixel positions of the body part. An aggregated body part position is determined based on the set of pixel positions. If the aggregated body part position is determined to correspond to a position associated with the object, a trigger signal is provided indicating an interaction event has occurred.

Event trigger based on region-of-interest near hand-shelf interaction

A system for refining an item identification model detects a triggering event at a platform. The system captures images of the item. The system extracts a set of features from at least one of the images. The system identifies the item based on the set of features. The system receives an indication that the item is not identified correctly. The system receives an identifier of the item. The system identifies the item based on the identifier of the item. The system feeds the identifier of the item and the images to the item identification model. The system retrains the item identification model to learn to associate the item to the images. The system updates the set of features based on the determined association between the item and the images.

System and method for refining an item identification model based on feedback

An object tracking system that includes a sensor and a tracking system. The sensor is configured to capture a frame of at least a portion of a rack within a global plane for a space. The tracking system is further configured to receive the frame, to determine a pixel location for the first person, and to determine a person is within the predefined zone associated with the rack. The tracking system is further configured to identify the plurality of items in a digital cart associated with the person, to identify an item from the digital cart associated with the first person, and to remove the identified item from the digital cart associated with the person.

Detecting and identifying misplaced items using a sensor array

A device configured to identify a first pixel location within a first plurality of pixels corresponding with an item in a first image and to apply a first homography to the first pixel location to determine a first (x,y) coordinate. The device is further configured to identify a second pixel location within a second plurality of pixels corresponding with the item in a second image and to apply a second homography to the second pixel location to determine a second (x,y) coordinate. The device is further configured to determine that the distance between the first (x,y) coordinate and the second (x,y) coordinate is less than or equal to the distance threshold value, to associate the first plurality of pixels and the second plurality of pixels with a cluster for the item, and to output the first plurality of pixels and the second plurality of pixels.

Item location detection using homographies

A device configured to capture a first overhead depth image of the platform using a three-dimensional (3D) sensor at a first time instance and a second overhead depth image of a first object using the 3D sensor at a second time instance. The device is further configured to determine that a first portion of the first object is within a region-of-interest and a second portion of the first object is outside the region-of-interest in the second overhead depth image. The device is further configured to capture a third overhead depth image of a second object placed on the platform using the 3D sensor at a third time instance. The device is further configured to capture a first image of the second object using a camera in response to determining that the first object is outside of the region-of-interest and the second object is within the region-of-interest for the platform.

Hand detection trigger for item identification

An object tracking system that includes a first sensor and a second sensor that are each configured to capture frames of at least a portion of a global plane for a space. The system is configured to identify a pixel location for a marker within a frame from the first sensor and to determine an (x,y) coordinate for the marker using a first homography. The system is further configured to identify a pixel location for a different marker in a frame from the second sensor and to determine an (x,y) coordinate for the marker using a second homography. The system is further configured to determine a distance difference between the computed distance between the (x,y) coordinates and an actual distance. The system is further configured to recompute the first homography and/or the second homography in response to determining that the distance difference exceeds a difference threshold level.

Growing community of inventors

Celina, TX, United States of America

Shahmeer Ali Mirza