Videomining Corporation

Satish Mummareddy

Rajeev Sharma

Namsoon Jung

Jeff Hershey

Hankyu Moon

Varij Saurabh

Priya Baboo

Emilio Schapira

Chris Ahlers

Akhil Kamat

The present invention is a method and system to provide correspondences between point-of-sale data registered at the store checkout and shopper behavior data observed at point-of-purchase through video analysis. The point-of-sale data include the list of shoppers and purchase items, and the shopper behavior data include the purchase events along with observed purchase items. The correspondence in the form of checkout shopper IDs matched to purchase event IDs is derived based on the algebraic constraint among the point-of-sale data and the purchase event data. Additional constraint based on shopper tracks and checkout/event times can also be incorporated to the correspondence problem. Uncertainties due to the video measurement can be systematically handled utilizing a Bayesian model.

Method and system for finding correspondence between point-of-sale data and customer behavior data

The present invention is a method and system for measuring viewership of people for a displayed object. The displayed object can be specific in-store marketing elements, such as static signage, POP displays, and other forms of digital media, including retail TV networks and kiosks. In the present invention, the viewership comprises impression level, impression count of the viewers, such as how many people actually viewed said displayed object, average length of impression, distribution of impressions by time of day, and rating of media effectiveness based on audience response. The viewership of people is performed automatically based on the 3-dimensional face pose estimation of the people, using a plurality of means for capturing images and a plurality of computer vision technologies on the captured visual information. The present invention distinguishes viewers from passers-by among the plurality of persons in the vicinity of the displayed object, by counting the number of viewers who actually viewed the displayed object vs. passers-by who may appear in the vicinity of the displayed object but do not actually view the displayed object, using the 3-dimensional face pose estimation and a novel usage of a plurality of computer vision technologies.

Method and system for measuring viewership of people for displayed object

The present invention is a method and system for forecasting the demographic characterization of customers to help customize programming contents on each means for playing output of each site of a plurality of sites in a media network through automatically measuring, characterizing, and estimating the demographic information of customers that appear in the vicinity of each means for playing output. The analysis of demographic information of customers is performed automatically based on the visual information of the customers, using a plurality of means for capturing images and a plurality of computer vision technologies on the visual information. The measurement of the demographic information is performed in each measured node, where the node is defined as means for playing output. Extrapolation of the measurement characterizes the demographic information per each node of a plurality of nodes in a site of a plurality of sites of a media network. The forecasting and customization of the programming contents is based on the characterization of the demographic information.

Method and system for automatically measuring and forecasting the demographic characterization of customers to help customize programming contents in a media network

The present invention is a method and system for optimizing the observation and annotation of complex human behavior from video sources by automatically detecting predefined events based on the behavior of people in a first video stream from a first means for capturing images in a physical space, accessing a synchronized second video stream from a second means for capturing images that are positioned to observe the people more closely using the timestamps associated with the detected events from the first video stream, and enabling an annotator to annotate each of the events with more labels using a tool. The present invention captures a plurality of input images of the persons by a plurality of means for capturing images and processes the plurality of input images in order to detect the predefined events based on the behavior in an exemplary embodiment. The processes are based on a novel usage of a plurality of computer vision technologies to analyze the human behavior from the plurality of input images. The physical space may be a retail space, and the people may be customers in the retail space.

Method and system for optimizing the observation and annotation of complex human behavior from video sources

The present invention is a method and system for producing a set of ratings for out-of-home media based on the measurement of behavior patterns and demographics of the people in a digital media network. The present invention captures a plurality of input images of the people in the vicinity of sampled out-of-home media in a digital media network by a plurality of means for capturing images, and tracks each person. The present invention processes the plurality of input images in order to analyze the behavior and demographics of the people. The present invention aggregates the measurements for the behavior patterns and demographics of the people, analyzes the data, and extracts characteristic information based on the estimated parameters from the aggregated measurements. Finally, the present invention calculates a set of ratings based on the characteristic information. The plurality of computer vision technologies can comprise face detection, person tracking, body parts detection, and demographic classification of the people, on the captured visual information of the people in the vicinity of the out-of-home media.

Method and system for rating of out-of-home digital media network based on automatic measurement

The present invention is a system and method for determining the hierarchical purchase decision process of consumers in front of a product category. The decision path of consumers is obtained by combining behavior data with the category layout and transaction data based on observed actual in-store purchase behavior using a set of video cameras and software for extracting sequence and timing of each consumer's decision process. A hierarchical decision tree structure comprises nodes and edges, wherein a node represents the state-of-mind of the consumer, the number of nodes is predefined, and an edge represents the transition of the decision. The decisions for each product group are captured down to the product attribute level and analyzed by demographic group. The outcome provides relative importance of each product attribute in the purchase decision process, and helps retailers and manufacturers to evaluate the layout of the category and customize it for key segment.

Method and system for building a consumer decision tree in a hierarchical decision tree structure based on in-store behavior analysis

The present invention is a method and system for analyzing shopping behavior by associating RFID data, such as tracking data by the RFID tag identifications, with video-based behavior and segmentation data, such as behavior analysis and demographic composition analysis of the customers, utilizing a plurality of means for sensing and using RFID tags, a plurality of means for capturing images, and a plurality of computer vision technologies. In the present invention, the association can further comprise the association of the RFID with the transaction data or any time-based measurement in the retail space. The analyzed shopping behavior in the present invention helps people to better understand business elements in a retail space. It is one of the objectives of the present invention to provide an automatic video-based segmentation of customers in the association with the RFID based tracking of the customers, based on a novel usage of a plurality of means for capturing images and a plurality of computer vision technologies on the captured visual information of the people in the retail space. The plurality of computer vision technologies can comprise face detection, person tracking, body parts detection, and demographic classification of the people, on the captured visual information of the people in the retail space.

Method and system for analyzing shopping behavior in a store by associating RFID data with video-based behavior and segmentation data

The present invention is a method and system for segmenting a plurality of persons in a physical space based on automatic behavior analysis of the persons in a preferred embodiment. The behavior analysis can comprise a path analysis as one of the characterization methods. The present invention applies segmentation criteria to the output of the video-based behavior analysis and assigns segmentation label to each of the persons during a predefined window of time. In addition to the behavioral characteristics, the present invention can also utilize other types of visual characterization, such as demographic analysis, or additional input sources, such as sales data, to segment the plurality of persons in another exemplary embodiment. The present invention captures a plurality of input images of the persons in the physical space by a plurality of means for capturing images. The present invention processes the plurality of input images in order to understand the behavioral characteristics, such as shopping behavior, of the persons for the segmentation purpose. The processes are based on a novel usage of a plurality of computer vision technologies to analyze the visual characterization of the persons from the plurality of input images. The physical space may be a retail space, and the persons may be customers in the retail space.

Method and system for segmenting people in a physical space based on automatic behavior analysis

The present invention is a method and system for automatically analyzing a category in a plurality of the categories in a physical space based on the visual characterization, such as behavior analysis or segmentation, of the persons with regard to the category. The present invention captures a plurality of input images of the persons in the category by a plurality of means for capturing images. The present invention processes the plurality of input images in order to understand the shopping behavior of the persons with the sub-categories of the category and analyzes the level of engagement and decision process at the sub-category level. The processes are based on a novel usage of a plurality of computer vision technologies to analyze the visual characterization of the persons from the plurality of input images. The physical space may be a retail space, and the persons may be customers in the retail space.

Method and system for automatically analyzing categories in a physical space based on the visual characterization of people

The present invention is a method and system for automatically determining the trip of people in a physical space, such as retail space, by capturing a plurality of input images of the people by a plurality of means for capturing images, processing the plurality of input images in order to track the people in each field of view of the plurality of means for capturing images, mapping the trip on to the coordinates of the physical space, joining the plurality of tracks across the multiple fields of view of the plurality of means for capturing images, and finding information for the trip of the people based on the processed results from the plurality of tracks. The trip information can comprise coordinates of the people's position and temporal attributes, such as trip time and trip length, for the plurality of tracks. The physical space may be a retail space, and the people may be customers in the retail space. The trip information can provide key measurements along the entire shopping trip, from entrance to checkout, that deliver deeper insights about the trip as a whole.

Method and system for automatic analysis of the trip of people in a retail space using multiple cameras

The present invention is a method and system for automatically analyzing the behavior of a person and a plurality of persons in a physical space based on measurement of the trip of the person and the plurality of persons on input images. The present invention captures a plurality of input images of the person by a plurality of means for capturing images, such as cameras. The plurality of input images is processed in order to track the person in each field of view of the plurality of means for capturing images. The present invention measures the information for the trip of the person in the physical space based on the processed results from the plurality of tracks and analyzes the behavior of the person based on the trip information. The trip information can comprise coordinates of the person's position and temporal attributes, such as trip time and trip length, for the plurality of tracks. The physical space may be a retail space, and the person may be a customer in the retail space. The trip information can provide key measurements as a foundation for the behavior analysis of the customer along the entire shopping trip, from entrance to checkout, that deliver deeper insights about the customer behavior. The focus of the present invention is given to the automatic behavior analytics applications based upon the trip from the extracted video, where the exemplary behavior analysis comprises map generation as visualization of the behavior, quantitative category measurement, dominant path measurement, category correlation measurement, and category sequence measurement.

Method and system for analyzing shopping behavior using multiple sensor tracking

The present invention is a method and system for characterizing physical space based on automatic demographics measurement, using a plurality of means for capturing images and a plurality of computer vision technologies. The present invention is called demographic-based retail space characterization (DBR). Although the disclosed method is described in the context of retail space, the present invention can be applied to any physical space that has a restricted boundary. In the present invention, the physical space characterization can comprise various types of characterization depending on the objective of the physical space, and it is one of the objectives of the present invention to provide the automatic demographic composition measurement to facilitate the physical space characterization. The demographic classification and composition measurement of people in the physical space is performed automatically based on a novel usage of a plurality of means for capturing images and a plurality of computer vision technologies on the captured visual information of the people in the physical space. The plurality of computer vision technologies can comprise face detection, person tracking, body parts detection, and demographic classification of the people, on the captured visual information of the people in the physical space.

Method and system for characterizing physical retail spaces by determining the demographic composition of people in the physical retail spaces utilizing video image analysis

The present invention is a method and system for forecasting the behavioral characterization of customers to help customize programming contents on each means for playing output of each site of a plurality of sites in a media network through automatically measuring, characterizing, and forecasting the behavioral information of customers that appear in the vicinity of each means for playing output. The analysis of behavioral information of customers is performed automatically based on the visual information of the customers, using a plurality of means for capturing images and a plurality of computer vision technologies on the visual information. The measurement of the behavioral information is performed in each measured node, where the node is defined as means for playing output. Extrapolation of the measurement characterizes the behavioral information per each node of a plurality of nodes in a site of a plurality of sites of a media network. The forecasting and customization of the programming contents is based on the characterization of the behavioral information.

Method and system for automatically measuring and forecasting the behavioral characterization of customers to help customize programming contents in a media network

The present invention is a method and system for recognizing employees among the people in a physical space based on automatic behavior analysis of the people in a preferred embodiment. The present invention captures a plurality of input images of the people in the physical space by a plurality of means for capturing images. The present invention processes the plurality of input images in order to understand the behavioral characteristics of the people for the employee recognition purpose. The behavior analysis can comprise a path analysis as one of the characterization methods. The path analysis collects a plurality of trip information for each tracked person during a predefined window of time. The trip information can comprise spatial and temporal attributes, such as coordinates of the person's position, trip time, trip length, and average velocity for each of the plurality of trips. Based on the employee recognition criteria applied to the trip information, the present invention distinguishes employees from non-employees during a predefined window of time. The processes are based on a novel usage of a plurality of computer vision technologies to analyze the behavior of the people from the plurality of input images.

Method and system for recognizing employees in a physical space based on automatic behavior analysis

The present invention is a method and system for automatically changing the advertisement contents on a means for displaying the contents based on a customer's behavior or a group of customers' behavior in a retail store. The analysis of behavioral pattern is performed automatically and in real-time based on the visual information from the shopping and walkthrough history of a customer or a group of customers, using arrays of sensing devices, such as a plurality of means for capturing images, and a plurality of computer vision technologies on the visual information.

Growing community of inventors

Washington, DC, United States of America

Satish Mummareddy