International Business Machines Corporation

Sensormatic Electronics Corporation

Joseph Celi, Jr

Gopi Subramanian

Victor S Moore

Olgierd Stanislaw Pieczul

Mark Alexander McGloin

Jonathan Mark Wagner

Brett J Gavagni

Thomas Edward Creamer

Michael C Stewart

Roger Louie

Ashok C Mammen

Peeyush Jaiswal

Neil A Katz

Margarita Zabolotskaya

Bharat B Purohit

Michael Bender

Wendi L Nusbickel

David Jaramillo

Mary Ellen Zurko

Glen R Walters

Gordan G Greenlee

Bruce D Lucas

Miriam Margarita Celi

Scott Lee Winters

Michael Richard Cooper

David E Reich

Amado Nassiff

Steven Gary Woodward

Joseph J Kubik

William Carleton Wimer, Ii

Kyriakos L Leontiades

Robert Selby Sielken

Michelangelo Palella

Frank J Castaneda

Robert Allen Rose

Jeffrey R Rendlen

David Lee Griffin

Harshita Nersu

Cory Olog Root

Michael Paolella

Harsh Gaurangbhai Motka

William Dubois

Yash Chaturvedi

Milton R Navarro

Karen Neidlinger Momenee

Natasha Mishell Astudillo

John P Coffey

Gordon D Webster

Leo Leontiades

Anne M Aldous

Christopher James Martinez

A system may be configured to generate real-time analytics using machine learning and computer vision. In some aspects, the system may receive a storage structure video frame and an interaction video frame, determine a storage structure inference based on the storage structure video frame, and determine an interaction inference based on the interaction video frame. Further, the system may determine that the storage structure inference and the interaction inference correspond to a common time period and common location, and generate analytics information based on the storage structure inference and the interaction inference.

Method and system to provide real time interior analytics using machine learning and computer vision

The present disclosure relates to a system for determining anomalous behavior. The system comprises a processing device that is configured to generate first segmentation information and second segmentation information from the video feed, the first segmentation information corresponding to a first potential occupant and the second segmentation information corresponding to a second potential occupant. The processing device further configured to determine that the first segmentation information and the second segmentation information correspond to actual occupants based on the first segmentation information and the second segmentation information, and generate a first bounding box associated with the first segmentation information and a second bounding box associated with the second segmentation information. The processing device further configured to determine an intersection over union (IOU) region for the first bounding box and the second bounding box, identify anomalous behavior based on the IOU region, and generate one or more alerts identifying the anomalous behavior.

System and method to determine anomalous behavior

Disclosed herein are apparatuses and methods for tracking occupancy count in an environment based on detected movements and dwell time. An implementation may comprise detecting a person in a plurality of image frames captured by a camera. The implementation may comprise tracking a movement of the person and determining that the person has crossed a boundary within the plurality of image frames. The implementation may comprise identifying a path of the person based on the tracked movement and may comprise calculating a dwell time in response to determining that a difference between the identified path and at least one target path associated with entering or exiting the environment exceeds a threshold difference. The implementation may comprise comparing the dwell time to a target dwell time, and updating and storing the occupancy count of the environment in response to determining that the dwell time is less than the target dwell time.

Systems and methods for vision system tracking in an environment based on detected movements and dwell time

Example aspects include a method, apparatus and computer-readable medium of determining losses at a point of sale (POS) device, comprising receiving, by a processor from an imaging device, a video feed of a scanning area. The aspects further include detecting, by the processor, an entry of an item into the scanning area. Additionally, the aspects further include identifying, by the processor, one or more motion parameters of the item. Additionally, the aspects further include determining, by the processor, a dwell-time for the item based at least on the one or more motion parameters. Additionally, the aspects further include identifying, by the processor, a scan time anomaly for the item. Additionally, the aspects further include outputting a notification, by the processor, indicating a suspicious activity for the item, wherein the notification indicating the suspicious activity is based on the scan time anomaly.

Scan avoidance prevention system

Disclosed herein are apparatuses and methods for iteratively mapping a layout of an environment. The implementations include receiving a visual stream from a camera installed in the environment, wherein the visual stream depicts a view of the environment, and wherein positional parameters of the camera and dimensions of the environment are set to arbitrary values. The implementations include monitoring a plurality of persons in the visual stream. For each person in the plurality of persons, the implementations further includes identifying a respective path that the person moves along in the view, updating the dimensions of the environment captured in the view, based on an estimated height of the person and movement speed along the respective path, and updating the positional parameters of the camera based on the updated dimensions of the environment. The implementations further includes mapping a layout of the environment captured in the view of the camera.

Layout mapping based on paths taken in an environment

Aspects of the present disclosure include methods, systems, and non-transitory computer readable media for receiving an initial image of the area, receiving an updated image of the area, detecting at least one of a person in the area or a movement in the area based on the updated image, comparing the updated image and the initial image to identify a change in the area, and determining a presence or an absence of the occlusion based on the change in the area and the at least one of the person or the movement in the area.

Methods and apparatuses for occlusion detection

Example implementations include a method, apparatus and computer-readable medium for person detection by a computer device, comprising receiving a plurality of video frames from a camera located in an environment, wherein the video frames include a depiction of a person in the environment. The implementations further include identifying attributes of the person from the video frames and comparing the attributes with attribute entries in a database of suspicious persons, wherein the database of suspicious persons includes attributes of a plurality of persons associated with an alert in the environment. The implementations include retrieving a profile indicative of an alert history of the suspicious person in response to determining a match between the attributes of the person and an entry in the database corresponding to a suspicious person, and transmitting, to a second computer device, the retrieved profile and a notification that indicates that the suspicious person is in the environment.

Person-of-interest (POI) detection

Aspects of the present disclosure relate generally to a vision system that detects persons exiting an environment, comprising detecting, by a processor using a plurality of image frames from at least one sensor, persons that exited the environment during a first period of time. The vision system further determines, by the processor, an exit count for the first period of time and retrieves, from a database, historical egress data comprising a detected historic exit count and a corrected historic exit count of the environment for a second period of time corresponding to the first period of time. The vision system calculates an error rate for the second period of time based on a ratio of the detected historic exit count and the corrected historic exit count, determines and stores a corrected exit count for the first period of time by adjusting the exit count using the error rate.

Systems and methods for object detection in an environment

Disclosed herein are apparatuses and methods for detecting a slip, trip or fall event in an environment and sending an alert of the event. An implementation may comprise detecting, in image frames captured by a camera, when an object enters a region of interest, tracking movements of the object, and determining that the object is in a fall-zone. The fall-zone may be specified by a set of line segments, each line segment being defined by points that lie in the region of interest. The implementation may further comprise, when the object is in the fall-zone, recording the position of the object, detecting a slip, trip or fall event when the object transitions from a first position above a predetermined height threshold to a second position below the predetermined height threshold, a pose of the object indicates that the object is lying down, or the object transitions from a vertical to horizontal pose, and sending the alert.

Systems and methods for detecting a slip, trip or fall

A system may be configured to perform enhanced detection of occluded objects in a multiple object detection system. In some aspects, the system may detect a plurality of current objects in a current video frame, generate initial object identifier mapping information assigning a candidate object identifier to a first object of the plurality of current objects, and detect a multi-object tracking (MOT) inaccuracy condition. Further, the system may generate a current fingerprint corresponding to the first object of the plurality of current objects, identify an assignment error based on comparing the current fingerprint to historic fingerprints associated with a plurality of historic objects detected in a previous frame, generate updated object identifier mapping information based on the assignment error, and displaying the current video frame based on the updated object identifier mapping information.

Multi-pass object tracking system utilizing single object tracking in a multi object tracking use case for higher accuracy

Disclosed herein are apparatuses and methods for auto-configuring a region of interest (ROI) associated with a camera. In one implementation, a method comprises receiving image frames from a camera installed in the environment, wherein the ROI is located within the view of the image frames. The method includes tracking a plurality of persons in the image frames and determining a respective trajectory of movement for each person of the plurality of persons. The method further includes comparing each of the respective trajectories to one another and identifying, based on the comparing, a common trajectory shared by more than one person of the plurality of persons, wherein the common trajectory is not fully encompassed in the ROI. The method additionally includes updating the ROI to encompass the common trajectory, and includes configuring the updated ROI to be associated with new image frames from the camera.

Auto-configuring a region of interest (ROI) associated with a camera

A system may be configured to reliably identify, track, and collect data on a person of interest. In some aspects, the system may detect a facial representation within an image, generate a bounding box corresponding to the facial representation, determine an enhanced facial representation based at least in part on the bounding box and an image enhancement pipeline, and extract a plurality of facial features from the enhanced facial representation. Further, the system may determine quality information based on the plurality of facial features, determine that the quality information is greater than a threshold, and store the plurality of facial features in a data structure. In some aspects, the facial features may be used to identify red shoppers and green shoppers in a controlled environment.

Method and system for identifying, tracking, and collecting data on a person of interest

A system may be configured to perform enhanced detection of occluded objects in a multiple object detection system. In some aspects, the system may detect a plurality of current objects in a current video frame, generate initial object identifier mapping information assigning a candidate object identifier to a first object of the plurality of current objects, and detect a multi-object tracking (MOT) inaccuracy condition. Further, the system may generate a current fingerprint corresponding to the first object of the plurality of current objects, identify an assignment error based on comparing the current fingerprint to historic fingerprints associated with a plurality of historic objects detected in a previous frame, generate updated object identifier mapping information based on the assignment error, and display the current video frame based on the updated object identifier mapping information.

Enhancing detection of occluded objects in a multiple object detection system

A system may be configured to accurately track shelf activity in real-time with support for dynamic shelf size, configuration, and item containment. In some aspects, the system may parse regions of a video frame to determine a region of interest representation corresponding to a physical location (e.g., a shelf compartment), determine an enhanced region of interest representation based at least in part on the region of interest representation and an image enhancement pipeline, determine edge information of one or more objects based on the enhanced region of interest representation, compare a reference representation of the physical location to the edge information, and determine the amount of available space for the physical location based on the comparing.

Real time tracking of shelf activity supporting dynamic shelf size, configuration and item containment

A system may be configured to perform object counting in high volume traffic. In some aspects, the system may determine object detection information defining an absence of an object within the region of interest in a current video frame, and determine that a location associated with the object is within a placeholder-eligible area within the region of interest based on first tracker information determined in a previous video frame. Further, the system may determine second tracker information including a bounding representation for the object based at least in part on the location and object detection information, and determine, based on the second tracker information, an object count representing a number of the objects that have entered the region of interest and/or a number of the objects that have exited the region of interest.

Object counting system for high volume traffic

Growing community of inventors

Boca Raton, FL, United States of America

Joseph Celi, Jr