Videomining Corporation

Other

Advanced Interfaces, LLC

Vmine Image Tech Co., Ltd LLC

Namsoon Jung

Rajeev Sharma

Hankyu Moon

Satish Mummareddy

Varij Saurabh

Jeff Hershey

Priya Baboo

Joonhwa Shin

Emilio Schapira

Vijaykrishnan Narayanan

Kevin Maurice Irick

Nils Oliver Krahnstoever

Nils Krahnstoever

The method and system evaluates the effectiveness of a display location within a store based on a behavioral response analysis of shoppers in the vicinity. The effectiveness of a display location is measured by tracking shoppers in-store, extracting and processing shopper attributes, and extracting metrics based on the processed attributes. The metrics of one location is compared to the metrics of another location to determine overall effectiveness.

Method and system for measuring in-store location effectiveness based on shopper response and behavior analysis

The present invention is a method and system for evaluating relative effectiveness of content in a digital signage network, using a series of measurements to compare and evaluate different contents. The measurements are performed on a plurality of input images of the people captured by a plurality of means for capturing images in the vicinity of at least a digital signage. The present invention is a content rating system based on how the audience responds to the digital content in the intended context of digital media network in-store or in other public viewing areas. The present invention allows the measurement of the response to the content for different segments, further allowing for better planning of the content and optimizing the advertising dollars.

Method and system for evaluating content for digital displays by measuring viewer responses by demographic segments

The present invention is a method and system for capturing a dataset over time that represents the response of shoppers to a set of marketing and merchandising stimuli or 'elements'. The response includes exposure metrics, engagement metrics and conversion metrics for different shopper segments. The shopper segments may be defined by demographics such as gender, age and ethnicity or by the type of trip, such as a quick trip or a fill-up trip. The system comprises a plurality of means for capturing images, such as cameras, covering the area of interest in the vicinity of the marketing or merchandising element. The method comprises automated and semi-automated analysis of the video to extract the shopper behavior and demographics data for computing the defined metrics. The captured data can be further combined with information such a promotions and advertisement outside the store to further enhance the applications of the data collected in-store from the invention. The captured shopper data can be used for many applications such as comparison of the effectiveness of different marketing elements or the relative effectiveness of different types of promotions, variation of the relative effectiveness of different marketing elements over time or differences between the relative effectiveness of a marketing element for different demographic segments.

Method and system for collecting shopper response data tied to marketing and merchandising elements

The present invention is a method and system for detecting shopping groups based on the dynamic relation between shoppers' trajectories. First, shopper trajectories are generated using video images or positioning devices. Then, group behavior features are extracted from a candidate shopping group trajectory pair. The group behavior features of a given pair of trajectories are typically the changes in positional differences and the average speed of these trajectories. From a model of shopping group behavior, a given pair of candidate shopping group trajectories is analyzed to determine the group score&#x2014;the likelihood of the pair of shoppers indeed belonging to the same group. Lastly, the system utilizes graph segmentation framework to find clusters where trajectories belonging to each cluster have tight group scores with each other.

Method and system for detecting shopping groups based on trajectory dynamics

The present invention is a system and method for measuring effectiveness of a marketing campaign on digital signage on many different signage networks, by measuring the efficiency of the campaign at reaching targeted audience and the effectiveness of conveying the message. This invention provides a solution to the challenges created by wide variety of measurements and lack of accuracy. By using automated audience measurement, the current invention is able to collect, large, statistically significant data for analysis. Non-intrusive, computer based measurement also ensure that the data is free from any biases. The media content rating system will provide a quantitative measure of how many people did the campaign reach and what effect did it have. The data will be available at the aggregate level, at network level and down to the screen level.

Method and system for measuring effectiveness of a marketing campaign on digital signage

The present invention is a method and system for rating in-store media elements based on the measurement for behavior patterns and demographics of the audience in the vicinity of the media element, where a plurality of input images of the audience are captured by at least a means for capturing images in the vicinity of the media element. The input images are processed by automated video analytic algorithms in order to measure the behavior patterns and demographics of each person in the audience tied to the media element. The measurements for the behavior patterns and demographics of the audience from the target media elements are aggregated. A set of ratings for the media element is calculated based on the measurements. The measured media elements are sampled, and the audience measurement for the sample is extrapolated to other media elements for in-store locations or for a group of stores. The ratings are converted into a standardized output. The standardized output facilitates comparisons among a plurality of media elements. The rating can be used to plan media types in the media element for reaching targeted demographics.

Method and system for rating in-store media elements

The present invention is a method and system for automatically recognizing which products a shopper intends to find or purchase based on the shopper's trajectory in a retail aisle. First, the system detects and tracks the person to generate the trajectory of the shopper. Then some of the dynamic features are extracted from the shopper trajectory. The shopper trajectory features of a given trajectory are typically the positions, the motion orientations, and speeds at each point of the trajectory. A shopper behavior model is designed based on some of the primitive actions of shoppers. The last step of the method is to analyze a given shopper trajectory to estimate the shopper's intention. The step either utilizes decision rules based on the extracted shopper trajectory features, or utilizes a trained Hidden Markov Model, to estimate the progression of the primitive actions from the trajectory. The decoded progression of the shopper behavior states is then interpreted to finally determine the shopper's intention.

Method and system for recognizing the intentions of shoppers in retail aisles based on their trajectories

The present invention is a system and method for performing ethnicity classification based on the facial images of people, using multi-category decomposition architecture of classifiers, which include a set of predefined auxiliary classifiers that are specialized to auxiliary features of the facial images. In the multi-category decomposition architecture, which is a hybrid multi-classifier architecture specialized to ethnicity classification, the task of learning the concept of ethnicity against significant within-class variations, is handled by decomposing the set of facial images into auxiliary demographics classes; the ethnicity classification is performed by an array of classifiers where each classifier, called an auxiliary class machine, is specialized to the given auxiliary class. The facial image data is annotated to assign the age and gender labels as well as the ethnicity labels. Each auxiliary class machine is trained to output both the given auxiliary class membership likelihood and the ethnicity likelihoods. Faces are detected from the input image, individually tracked, and fed to all the auxiliary class machines to compute the desired auxiliary class membership and ethnicity likelihood outputs. The outputs from all the auxiliary class machines are combined in a manner to make a final decision on the ethnicity of the given face.

Method and system for determining ethnicity category of facial images based on multi-level primary and auxiliary classifiers

The present invention provides a comprehensive method to design an automatic media audience measurement system that can estimate the site-wide audience of a media of interest (e.g., the site-wide viewership of a target display) based on the measurements of a subset of the actual audience sampled from a limited space in the site. This invention enables (1) the placement of sensors in optimal positions for the viewership data measurement and (2) the estimation of the site-wide viewership of the target display by performing the viewership extrapolation based on the sampled viewership data. The viewership extrapolation problem is formulated in a way that the time-varying crowd dynamics around the target display is an important decisive factor as well as the sampled viewership data at a given time in yielding the estimated site-wide viewership. To solve this problem, the system elements that affect the viewership&#x2014;site, display, crowd, and audience&#x2014;and their relationships are first identified in terms of the visibility, the viewership relevancy, and the crowd occupancy. The optimal positions of the sensors are determined to cover the maximum area of the viewership with high probabilities. The viewership extrapolation function is then modeled and learned from the sampled viewership data, the site-wide viewership data, and the crowd dynamics measurements while removing the noise in the sampled viewership data using the viewership relevancy of the measurements to the target display.

Method and system for media audience measurement by viewership extrapolation based on site, display, and crowd characterization

The present invention is a system, method, and apparatus for determining the impact of crowding on retail performance based on a measurement for behavior patterns of people in a store area. The present invention captures a plurality of input images of the people by at least a means for capturing images, such as cameras, in the store area. In the captured plurality of input images, each person's shopping path is detected by a video analytics-based tracking algorithm. A subset of the people is identified as a crowd in the store area. In relation to the crowd, the behavior patterns of the target person are measured. After aggregating the measurements for the behavior patterns over a predefined window of time, the present invention can calculate a crowd index and a crowd impact index for the store area based on the measurements. A crowd index shows the level of crowd density in the store area caused by a crowd, including traffic count of the crowd in the store area. A crowd impact index comprises a traffic count of the target people who make trips to the store area and a shopping time index, such as average shopping time changes of the target people, in relation to a crowd in the measured store area.

Method and system for determining the impact of crowding on retail performance

The present invention is a method and system for optimizing the observation and annotation of complex human behavior from video sources by automatically detecting predefined events based on the behavior of people in a first video stream from a first means for capturing images in a physical space, accessing a synchronized second video stream from a second means for capturing images that are positioned to observe the people more closely using the timestamps associated with the detected events from the first video stream, and enabling an annotator to annotate each of the events with more labels using a tool. The present invention captures a plurality of input images of the persons by a plurality of means for capturing images and processes the plurality of input images in order to detect the predefined events based on the behavior in an exemplary embodiment. The processes are based on a novel usage of a plurality of computer vision technologies to analyze the human behavior from the plurality of input images. The physical space may be a retail space, and the people may be customers in the retail space.

Method and system for optimizing the observation and annotation of complex human behavior from video sources

The present invention is a method and system for producing a set of ratings for out-of-home media based on the measurement of behavior patterns and demographics of the people in a digital media network. The present invention captures a plurality of input images of the people in the vicinity of sampled out-of-home media in a digital media network by a plurality of means for capturing images, and tracks each person. The present invention processes the plurality of input images in order to analyze the behavior and demographics of the people. The present invention aggregates the measurements for the behavior patterns and demographics of the people, analyzes the data, and extracts characteristic information based on the estimated parameters from the aggregated measurements. Finally, the present invention calculates a set of ratings based on the characteristic information. The plurality of computer vision technologies can comprise face detection, person tracking, body parts detection, and demographic classification of the people, on the captured visual information of the people in the vicinity of the out-of-home media.

Method and system for rating of out-of-home digital media network based on automatic measurement

The present invention is a method and system for measuring a set of shopper behavior metrics that represent the strength of a product category or a group of categories in the performance of a store area. A set of rating parameters are defined in order to provide a unified and standardized rating system. The rating system represents the effectiveness of the product category in a store area. The metrics are defined in a manner that is normalized so that they can be used across different types of product categories. The datasets are measured per category or group of categories over time to identify how the strength has varied over time, and to monitor trends in the category performance. The measured datasets are further analyzed based on various demographic groups and behavior segments. The analysis facilitates a better understanding of the strength of the category for different shopper segments, which in turn can be applied for developing better store area optimization strategies.

Method and system for rating the role of a product category in the performance of a store area

The present invention is a method and system for selecting and storing videos by applying semantically-meaningful selection criteria to the track sequences of the trips made by people in an area covered by overlapping multiple cameras. The present invention captures video streams of the people in the area by multiple cameras and tracks the people in each of the video streams, producing track sequences in each video stream. The present invention determines a first set of video segments that contains the trip information of the people, and compacts each of the video streams by removing a second set of video segments that do not contain the trip information of the people from each of the video streams. The present invention selects video segments from the first set of video segments based on predefined selection criteria for the statistical behavior analysis. The stored video data is an efficient compact format of video segments that contain the track sequences of the people and selected according to semantically-meaningful and domain-specific selection criteria. The final storage format of the videos is a trip-centered format, which sequences videos from across multiple cameras, and it can be used to facilitate multiple applications dealing with behavior analysis in a specific domain.

Method and system for efficient sampling of videos using spatiotemporal constraints for statistical behavior analysis

The present invention is a system and method for estimating the age of people based on their facial images. It addresses the difficulty of annotating the age of a person from facial image by utilizing relative age (such as older than, or younger than) and face-based class similarity (gender, ethnicity or appearance-based cluster) of sampled pair-wise facial images. It involves a unique method for the pair-wise face training and a learning machine (or multiple learning machines) which output the relative age along with the face-based class similarity, of the pairwise facial images. At the testing stage, the given input face image is paired with some number of reference images to be fed to the trained machines. The age of the input face is determined by comparing the estimated relative ages of the pairwise facial images to the ages of reference face images. Because age comparison is more meaningful when the pair belongs to the same demographics category (such as gender and ethnicity) or when the pair has similar appearance, the estimated relative ages are weighted according to the face-based class similarity score between the reference face and the input face.

Growing community of inventors

State College, PA, United States of America

Namsoon Jung