Machine learning methods and systems for tracking shoppers and interactions with items in a cashier-less store

Abstract

Method and systems are provided for processing actions in a store. One example method includes capturing sensor output from two or more sensors in a shopping environment. One sensor includes a first camera to capture scene data where the scene includes movement of a shopper in the store performing interaction with an item in the store. Another sensor includes a second camera capturing at least part of the scene from a different perspective. The method includes processing, by a processing entity associated with the store, at least one of the camera's output to generate feature data. The feature data is processed by one or more machine learning models to produce engineered feature data. The engineered feature data includes data relating to tracking skeletal movement of shopper. The method includes processing, by a processing entity associated with the store, the feature data including engineered feature data using said one or more machine learning models to produce a prediction that a take of the item has occurred by the shopper. The prediction is based on a characterization that said feature data or engineered feature data is labeled to infer that the movement of the shopper is interaction with the item that is classified as the take as having occurred.