System to determine user interaction with fixture

Abstract

Sensors in a facility obtain sensor data about a user's interaction with a fixture, such as a shelf. The sensor data may include images such as obtained from overhead cameras, weight changes from weight sensors at the shelf, and so forth. Based on the sensor data, one or more hypotheses that indicate the items and quantity may be determined and assessed. The hypotheses may be based on information such as the location of a user and where their hands are, weight changes, physical layout data indicative of where items are stowed, cart data indicative of what items are in the possession of the user, and so forth. A hypothesis having a greatest confidence value may be deemed to be representative of the user's interaction, and interaction data indicative of the item and quantity may be stored.