Carnegie Mellon University

Biometricore, Inc.

Bossa Nova Robotics Ip, Inc.

Hawxeye, Inc.

Bossa Nova Robotics

Marios Savvides

Uzair Ahmed

Chenchen Zhu

Fangyi Chen

Ran Tao

Dipan Kumar Pal

Yutong Zheng

Sreena Nallamothu

Vijayakumar Bhagavatula

Khoa Luu

Zhiqiang Shen

Karanhaar Singh

Khalid Harun

Kyle Neblett

Shreyas Venugopalan

Felix Juefei Xu

Sarjoun Skaff

Utsav Prabhu

Sungwon Park

Ajmal Thanikkal

Keshav Thirumalai Seshadri

Felix Juefei-Xu

Han Zhang

Nikhil Mohan

Gavriel Adler

Yu Kai Huang

Magesh Kannan

Yung-Hui Li

An Pang Lin

Sung Won Park

Christopher Palmer

Shayeree Sarkar

Unni Prasad

Chandrasekhar Bhagavatula

Kai Hu

Ahmed Uzair

Thi Hoang Ngan Le

Andy Lin

Abhishek Das

Thanh Hai Phan

Eddie Yu

Akshay Chawla

Douglas Toomey

Kahlid Harun

Jingu Heo

Yunghui Li

Chandrasekhar Bhagavatua

Raied Aljadaany

Disclosed herein is a system and method of identifying products on a retail shelf using a feature extractor trained to extract features from images of products on the shelf and output identifying information regarding the product in the product image. The extracted features are compared to extracted features in a feature gallery library and a best fit match is obtained. A product ID is then assigned to the image of the product and the assigned product ID is validated by matching the product ID with product identifying information extracted from a shelf label associated with the image of the product.

System and method for identifying products in a shelf management system

Disclosed here are various techniques for improving the testing and training of datasets comprising sequences of skeletal representations performing various actions. The dataset can be denoised by applying various techniques to determine noisy frames within each sequence and eliminating the sequences from the dataset when the number of noisy frames in the sequence is too large. In addition, the dataset may be augmented by various data augmentation techniques to manipulate the skeletal representations, after denoising.

Methods for improving datasets for skeleton-based action detection

Disclosed herein is a system and method for performing pose-correction on images containing objects within a scene, or the entire scene, to compensate for off-centered camera views. The system and method generates a more frontal view of the object or scene by applying planar homography by identifying corner endpoints of the object or the scene and repositioning the corner endpoints to provide a more frontal view. The pose-corrected scene may then be input to an object detector to determine a location of a bounding box of an object-of-interest which would be more accurate than a bounding box from the original off-centered image.

System and method for scene rectification via homography estimation

Disclosed herein is a system and method for improving the accuracy of an object detector when trained with a dataset having a significant number of missing annotations. The method uses a novel Background Recalibration Loss (BRL) which adjusts the gradient direction according to its own activation to reduce the adverse effect of error signals by replacing the negative branch of the focal loss with a mirror of the positive branch when the activation is below a confusion threshold.

System and method for solving missing annotation object detection

An automated inventory monitoring system includes an image capture module able to create an image of an aisle of a retail store. Product images and shelf label and peg label images are identified in the image and products are associated with product labels based on the positioning of the products with respect to the labels. Based on the association between labels and products, out-of-stock products are detected and reported to the retail store.

System and method for determining out-of-stock products

A method for increasing the confidence of a match between a test image and an image stored in a library database in which features are extracted from the test image and compared to features stored in the image database. If a match is determined, one or more transformations are performed on the test image to generate pose-altered images from which features are extracted and matched with pose-altered images in the database. The scores for the subsequent matchings can be aggregated to determine an overall probability of a match between the test image in an image in the library database.

Multiple hypothesis transformation matching for robust verification of object identification

Disclosed herein is a methodology for refining novel-class features in a few-shot learning scenario by fine-tuning the feature extractor by reducing both class-agnostic biases and class-specific biases. A distribution calibration module is used to reduce the class-agnostic bias by normalizing the overall feature distribution for novel classes and further reshaping the feature manifold for fast adaptation during fine-tuning. Selected sampling is used to reduce class-specific bias by augmenting more data for better estimation.

System and method for domain-agnostic bias reduction with selected sampling for few-shot learning

Disclosed herein is an improved few-shot detector which utilizes a dynamic semantic network which takes as input a language feature and generates trainable parameters for a visual network. The visual network takes a visual feature as input and generates a classification and localization of an object.

System and method for improved few-shot object detection using a dynamic semantic network

Disclosed herein is a system and method for novel neural architecture search using a random graph network backbone to facilitate the creation of an efficient network structure. The method utilizes reinforcement learning algorithms to build a complex relationship between intra-connections (i.e., links between blocks in a random graph network) and extra-connections (i.e., links among blocks across the random graphs network) for discovering an efficient random neural architecture.

System and method for the discovering efficient random neural networks

A method of training a deep neural network, such as would be used in facial recognition, includes training the deep neural network to normalize feature vectors to a learned value representing the radius of a multi-dimensional hypersphere using a convex augmentation of the primary loss function.

Convex feature normalization for face recognition

Disclosed herein is a method to disentangle linear-encoded facial semantics from facial images without external supervision. The method uses linear regression and sparse representation learning concepts to make the disentangled latent representations easily interpreted and manipulated. Generated facial images are decomposed into multiple semantic features and latent representations are extracted to capture interpretable facial semantics. The semantic features may be manipulated to synthesize photorealistic facial images by sampling along vectors representing the semantic features, thereby changing the associate semantics.

System and method for photorealistic image synthesis using unsupervised semantic feature disentanglement

During training of deep neural networks, a Copernican loss (L) is designed to augment a primary loss function, for example, a standard Softmax loss, to explicitly minimize intra-class variation and simultaneously maximize inter-class variation. Copernican loss operates using the cosine distance and thereby affects angles leading to a cosine embedding, which removes the disconnect between training and testing.

Discriminative cosine embedding in machine learning

Disclosed herein is an improved method for identifying images containing objects-of-interest from a large set of images. The method comprises mixing two or more of the images to create a grouped image and exposing the grouped image to an object detector trained on grouped images to make an initial determination that the grouped image was formed from at least one image containing an object-of-interest. The images which formed the grouped image are then exposed to regular object detectors to determine a classification of the object-of-interest.

Fast object search based on the cocktail party effect

An automated inventory monitoring system includes an image capture module able to create an image of an aisle of a retail store. Images of products labels are identified in the image and classified as shelf labels or peg labels. For shelf labels, an area of the shelf is defined and associated with the shelf label. Images of products are identified in the image and products on the shelf within an area associated with a shelf label are associated with the shelf label. Products located below a peg label are associated with the peg label. Based on the association between labels and products, out-of-stock products, plugs and spread may be detected and reported to the staff of the retail store.

System and method for associating products and product labels

Disclosed herein is a system and method of identifying misplaced products on a retail shelf using a feature extractor trained to extract features from images of products on the shelf and output identifying information regarding the product in the product image. The extracted features are compared to extracted features in a product library and a best fit is obtained. A misplaced product is identified if the identifying information produced by the feature extractor fails to match the identifying information associated with the best fit features from the product library.

System and method for identifying misplaced products in a shelf management system

Growing community of inventors

Wexford, PA, United States of America

Marios Savvides