Carnegie Mellon University

Bossa Nova Robotics

Sreena Nallamothu

Marios Savvides

Uzair Ahmed

Sarjoun Skaff

Ran Tao

Yutong Zheng

Nikhil Mohan

Magesh Kannan

Kai Hu

Abhishek Das

Ahmed Uzair

Disclosed herein is a system and method of identifying products on a retail shelf using a feature extractor trained to extract features from images of products on the shelf and output identifying information regarding the product in the product image. The extracted features are compared to extracted features in a feature gallery library and a best fit match is obtained. A product ID is then assigned to the image of the product and the assigned product ID is validated by matching the product ID with product identifying information extracted from a shelf label associated with the image of the product.

System and method for identifying products in a shelf management system

An automated inventory monitoring system includes an image capture module able to create an image of an aisle of a retail store. Product images and shelf label and peg label images are identified in the image and products are associated with product labels based on the positioning of the products with respect to the labels. Based on the association between labels and products, out-of-stock products are detected and reported to the retail store.

System and method for determining out-of-stock products

An automated inventory monitoring system includes an image capture module able to create an image of an aisle of a retail store. Images of products labels are identified in the image and classified as shelf labels or peg labels. For shelf labels, an area of the shelf is defined and associated with the shelf label. Images of products are identified in the image and products on the shelf within an area associated with a shelf label are associated with the shelf label. Products located below a peg label are associated with the peg label. Based on the association between labels and products, out-of-stock products, plugs and spread may be detected and reported to the staff of the retail store.

System and method for associating products and product labels

Disclosed herein is a system and method of identifying misplaced products on a retail shelf using a feature extractor trained to extract features from images of products on the shelf and output identifying information regarding the product in the product image. The extracted features are compared to extracted features in a product library and a best fit is obtained. A misplaced product is identified if the identifying information produced by the feature extractor fails to match the identifying information associated with the best fit features from the product library.

System and method for identifying misplaced products in a shelf management system

Disclosed herein is a system and method for augmenting data by generating a plurality of pose-altered images of an item from one or more 2D images of the item and using the augmented data to train a train a feature extractor. In other aspects of the invention, the trained feature extractor is used to enroll features extracted from images of new products in a library database of known products or to identify images of unknown products by matching features of an image of the unknown product with features stored in the library database.

Growing community of inventors

Pittsburgh, PA, United States of America

Sreena Nallamothu