Methods and systems for identifying actions of shoppers in stores in relation to items provided for sale in cashier-less transactions

Abstract

Methods and systems related to tracking activity in a store in association with a cashier-less environment are provided. One example method includes identifying actions in a store. The method includes sampling a shopping environment using one or more sensors that include at least one camera capable of providing depth sensing to produce image data of a scene that shows a shopper in the store and tracking data related to one or more limbs of the shopper in connection to an item. The method includes receiving output of the sampling as feature inputs to one or more machine learning models and deriving one or more label inferences of a behavior state of the shopper in connection with a state of the item. At least one processing entity associated with the store detects the state of the item to change from one as item handled by said shopper to one as item queued for purchase. The sensor data from said one or more sensors is used to identify a scenario that indicates the item chargeable to an electronic shopping cart of the shopper. In some examples, in addition to analyzing output from cameras, the analysis for identifying the item and/or the take of the item is augmented by processing data produced by sensors installed in or around shelfs or locations where the item or items are placed in the store.