Standard Cognition, Corp.

Jordan E Fisher

Daniel L Fischetti

Juan C Lasheras

John F Novak

Brandon L Ogle

Michael S Suswal

Warren Green

Kyle E Dorman

Kenneth S Kihara

Nicholas J Locascio

David Valdman

Alex Hu

Arpan Ghosh

Michael Huang

David Woollard

Atul Dhingra

Michele Toni

Nairwita Mazumder

Mohit Deep Singh

Systems and methods for tracking inventory items in an area of real space are disclosed. The method includes receiving a signal generated in dependence on sensors. The signal indicates a change to a portion of an image of an area of real space. The method includes, in response to receiving the signal, implementing a trained location detection model to determine, based on inputs, whether an inventory item identified in the portion of the image has changed a position in the area of real space. The method includes implementing a trained item classification model to determine a classification of the inventory item. The method includes updating an inventory database with inventory item data determined in dependence on the classification of the inventory item to provide an updated map of the area of real space as a result of the received signal indicating the change to the portion of the image.

Systems and methods for trigger-based updates to camograms for autonomous checkout in a cashier-less shopping

Systems and methods for tracking and re-identifying tracked subjects in an area of real space are disclosed. The method includes generating first and second reidentification feature vectors of a first subject identified from a first time interval from the respective first and second sequences of images as obtained from the first time interval. The method includes generating third and fourth reidentification feature vectors of a second subject identified from a second time interval from the respective first and second sequences of images. The method includes matching a second subject identified from a second time interval with the first subject identified from the first time interval when at least one of a first similarity score between the first and the third reidentification feature vectors and a second similarity score between the second and the fourth reidentification feature vectors is above a pre-defined threshold.

Machine learning-based re-identification of shoppers in a cashier-less store for autonomous checkout

Systems and techniques are provided for detecting gaze direction of subjects in an area of real space. The system receives a plurality of sequences of frames of corresponding fields of view in the real space. The system uses sequences of frames in a plurality of sequences of frames to identify locations of an identified subject and gaze directions of the subject in the area of real space over time. The system includes logic having access to a database identifying locations of items in the area of real space. The system identifies items in the area of real space matching the identified gaze directions of the identified subject.

Directional impression analysis using deep learning

Systems and techniques are provided for linking subjects in an area of real space with user accounts. The user accounts are linked with client applications executable on mobile computing devices. A plurality of cameras are disposed above the area. The cameras in the plurality of cameras produce respective sequences of images in corresponding fields of view in the real space. A processing system is coupled to the plurality of cameras. The processing system includes logic to determine locations of subjects represented in the images. The processing system further includes logic to match the identified subjects with user accounts by identifying locations of the mobile computing devices executing client applications in the area of real space and matching locations of the mobile computing devices with locations of the subjects.

Systems and methods to check-in shoppers in a cashier-less store

A method for identifying gestures in an area of real space is provided. The method including using a plurality of sensors to produce respective sequences of frames of corresponding fields of view in the area of real space, detecting subjects in the area of real space, using a sequence of frames produced by a corresponding sensor in the plurality of sensors in a first inference engine to identify inventory items carried by the detected subjects in the sequence of frames, using outputs of the first inference engine over a period of time in a second inference engine to identify gestures of the detected subjects, and storing the identified gestures in a database.

Inventory tracking system and method that identifies gestures of subjects holding inventory items

A method for tracking inventory items is provided. The method includes using sensors to produce respective sequences of images of corresponding fields of view, the field of view of each sensor overlapping with the field of view of at least one other sensor. The method includes dividing the area of real space into cells having volume and coordinates in a real space, and identifying inventory items and their locations in three dimensions using the sequences of images produced by at least two sensors having overlapping fields of view. The method includes, in response to the identification of the inventory items: (i) matching locations in three dimensions of identified inventory items with cells in the plurality of cells, and (ii) determining, for a particular inventory item matched with a particular cell, whether a count of the particular inventory item is below a threshold for re-stocking the particular inventory item.

Tracking inventory items in a store for identification of inventory items to be re-stocked and for identification of misplaced items

A method for identifying an inventory item is provided. The method includes using first one or more sensors to produce first one or more outputs indicative of an inventory item, using second one or more sensors to produce second one or more outputs indicative of the inventory item, generating, based on the first one or more outputs and independent of the second one or more outputs, a first confidence level of the inventory item being a first inventory item, generating, based on the second one or more outputs and independent of the first one or more outputs, a second confidence level of the inventory item being a second inventory item, and identifying, based on the first confidence level and the second confidence level, the inventory item as one of the first inventory item or the second inventory item.

Identifying inventory items using multiple confidence levels

A method of tracking subjects in an area. The method including receiving a plurality of sequences of images of corresponding fields of view in the area of real space, using a plurality of trained inference engines that process respective sequences of images in the plurality of sequences of images to locate features of subjects in the corresponding fields of view of the respective sequences, combining the located features from more than one of the trained inference engines which process respective sequences of images having overlapping fields of view to generate data locating subjects in three dimensions in the area of real space during identification intervals, and matching located subjects from a plurality of identification intervals to identify tracked subjects, including comparing located subjects with tracked subjects.

Systems and methods of implementing multiple trained inference engines to identify and track subjects over multiple identification intervals

Systems and techniques are provided for tracking inventory items in an area of real space. A plurality of cameras, or other sensors, produce respective sequences of images in corresponding fields of view in the real space. The field of view of each camera overlaps with the field of view of at least one other camera. The system is coupled to the plurality of cameras and uses the sequences of images produced by at least two cameras in the plurality of cameras to identify inventory events. The inventory event includes an item identifier, a location and a timestamp. A plurality of cells having coordinates in the area of real space are stored as a data set in the memory. The processing system calculates scores at a scoring time, for inventory items having locations matching particular cells using respective counts of inventory events.

Deep learning-based store realograms

Systems and techniques are provided for tracking puts and takes of inventory items by subjects in an area of real space. A plurality of cameras with overlapping fields of view produce respective sequences of images of corresponding fields of view in the real space. In one embodiment, the system includes first image processors, including subject image recognition engines, receiving corresponding sequences of images from the plurality of cameras. The first image processors process images to identify subjects represented in the images in the corresponding sequences of images. The system includes second image processors, including background image recognition engines, receiving corresponding sequences of images from the plurality of cameras. The second image processors mask the identified subjects to generate masked images. Following this, the second image processors process the masked images to identify and classify background changes represented in the images in the corresponding sequences of images.

Systems and methods for deep learning-based shopper tracking

Systems and techniques are provided for tracking inventory events in an area of real space. A plurality of sensors produce respective sequences of frames in corresponding fields of view in the real space. The field of view of each sensor overlaps with the field of view of at least one other sensor. The processing system uses the sequences of frames produced by sensors in the plurality of sensors to identify gestures by detected subjects in the area of real space and produce inventory events. The inventory events include a subject identifier identifying a detected subject, a gesture type of the identified gesture by the detected subject, an item identifier identifying an inventory item linked to the gesture by the detected subject, a location of the gesture represented by positions in three dimensions of the area of real space and a timestamp.

Product correlation analysis using deep learning

Systems and techniques are provided for monitoring status of subjects in an area of real space. The system includes a display having access to memory storing a first data set identifying locations of subjects in the area of real space over time, a second data set identifying item put and take events in the area of real space and a third data set identifying accounts matched to the subjects in the first data set. The system includes a graphical user interface logic that renders a map on the display of at least part of the area of real space including avatars having locations on the map corresponding to locations in the area of real space according to the first data set, and identifying respective statuses assigned to the subjects based on one or more of the first data set, the second data set and the third data set.

Growing community of inventors

San Francisco, CA, United States of America

Jordan E Fisher