Videomining Corporation

Other

Advanced Interfaces, LLC

The Penn State Research Foundation

Hyacinth Audio LLC

Vmine Image Tech Co., Ltd LLC

Rajeev Sharma

Namsoon Jung

Hankyu Moon

Satish Mummareddy

Jeff Hershey

Varij Saurabh

Joonhwa Shin

Donghun Kim

Emilio Schapira

Priya Baboo

Youngrock R Yoon

Vladimir Y Mariano

Vijaykrishnan Narayanan

Kuntal Sengupta

Richard Hirata

Krishna Botla

Nils Krahnstoever

Kevin Maurice Irick

Nils Oliver Krahnstoever

Chris Ahlers

Harish Raju

Pyush Agrawal

Mohammed Yeasin

Sanshzar Kettebekov

Akhil Kamat

The method and system evaluates the effectiveness of a display location within a store based on a behavioral response analysis of shoppers in the vicinity. The effectiveness of a display location is measured by tracking shoppers in-store, extracting and processing shopper attributes, and extracting metrics based on the processed attributes. The metrics of one location is compared to the metrics of another location to determine overall effectiveness.

Method and system for measuring in-store location effectiveness based on shopper response and behavior analysis

The present invention is a method and system for evaluating relative effectiveness of content in a digital signage network, using a series of measurements to compare and evaluate different contents. The measurements are performed on a plurality of input images of the people captured by a plurality of means for capturing images in the vicinity of at least a digital signage. The present invention is a content rating system based on how the audience responds to the digital content in the intended context of digital media network in-store or in other public viewing areas. The present invention allows the measurement of the response to the content for different segments, further allowing for better planning of the content and optimizing the advertising dollars.

Method and system for evaluating content for digital displays by measuring viewer responses by demographic segments

A method and system for creating an anonymous shopper panel based on multi-modal sensor data fusion. The anonymous shopper panel can serve as the same traditional shopper panel who reports their household information, such as household size, income level, demographics, etc., and their purchase history, yet without any voluntary participation. A configuration of vision sensors and mobile access points can be used to detect and track shoppers as they travel a retail environment. Fusion of those modalities can be used to form a trajectory. The trajectory data can then be associated with Point of Sale data to form a full set of shopper behavior data. Shopper behavior data for a particular visit can then be compared to data from previous shoppers' visits to determine if the shopper is a revisiting shopper. The shopper's data can then be aggregated for multiple visits to the retail location. The aggregated shopper data can be filtered using application-specific criteria to create an anonymous shopper panel.

Method and system for creating anonymous shopper panel using multi-modal sensor fusion

The present invention is a method and system for capturing a dataset over time that represents the response of shoppers to a set of marketing and merchandising stimuli or 'elements'. The response includes exposure metrics, engagement metrics and conversion metrics for different shopper segments. The shopper segments may be defined by demographics such as gender, age and ethnicity or by the type of trip, such as a quick trip or a fill-up trip. The system comprises a plurality of means for capturing images, such as cameras, covering the area of interest in the vicinity of the marketing or merchandising element. The method comprises automated and semi-automated analysis of the video to extract the shopper behavior and demographics data for computing the defined metrics. The captured data can be further combined with information such a promotions and advertisement outside the store to further enhance the applications of the data collected in-store from the invention. The captured shopper data can be used for many applications such as comparison of the effectiveness of different marketing elements or the relative effectiveness of different types of promotions, variation of the relative effectiveness of different marketing elements over time or differences between the relative effectiveness of a marketing element for different demographic segments.

Method and system for collecting shopper response data tied to marketing and merchandising elements

The present invention is a method and system for detecting shopping groups based on the dynamic relation between shoppers' trajectories. First, shopper trajectories are generated using video images or positioning devices. Then, group behavior features are extracted from a candidate shopping group trajectory pair. The group behavior features of a given pair of trajectories are typically the changes in positional differences and the average speed of these trajectories. From a model of shopping group behavior, a given pair of candidate shopping group trajectories is analyzed to determine the group score&#x2014;the likelihood of the pair of shoppers indeed belonging to the same group. Lastly, the system utilizes graph segmentation framework to find clusters where trajectories belonging to each cluster have tight group scores with each other.

Method and system for detecting shopping groups based on trajectory dynamics

A method and system for determining the hierarchical purchase decision process of a shopper in front of a product category in a retail store. Shopping consideration and the decision path the shopper can be obtained by combining behavior data with the category layout and transaction data. The hierarchical decision process can be determined based on observed actual in-store purchase behavior using a set of video cameras and processor implemented instructions for extracting the sequence and timing of the shopper's decision process. The hierarchical decision process, obtained by the clustering of shopper behavior data over multiple shopping trips, can not only identify the sequence of the decision, but can also quantify the volume of shopping trips at each level in the decision process and the amount of time spent by the shopper for making each decision. The decisions of the shopper for each product group can be captured down to the product attribute level. Such in-depth understanding can provide a picture of the relative importance of each product attribute in the purchase decision process, and can help retailers personalize targeted messaging to the shopper.

Personalized decision tree based on in-store behavior analysis

The present invention is a method and system to provide correspondences between point-of-sale data registered at the store checkout and shopper behavior data observed at point-of-purchase through video analysis. The point-of-sale data include the list of shoppers and purchase items, and the shopper behavior data include the purchase events along with observed purchase items. The correspondence in the form of checkout shopper IDs matched to purchase event IDs is derived based on the algebraic constraint among the point-of-sale data and the purchase event data. Additional constraint based on shopper tracks and checkout/event times can also be incorporated to the correspondence problem. Uncertainties due to the video measurement can be systematically handled utilizing a Bayesian model.

Method and system for finding correspondence between point-of-sale data and customer behavior data

A method and system for associating a mobile device carried by a shopper with Point-of-Sale (PoS) data from that shopper's transaction at a retail location. Specifically, the association can utilize face data that was associated with the mobile device to match face data that was generated by association with PoS data. The association with a mobile device can also include the calculation of a mobile trajectory to enable the generation of a shopper profile including shopper behavior. The mobile to PoS association can include tracking of the mobile device throughout a retail location to form a trajectory. The association of a face to a mobile device can utilize the device's MAC address and repeat visit analysis to determine a match. The association of a face to PoS data can utilize the aligning of event time series based on a dynamic time disparity to match checkout events with PoS events.

Association of mobile device to retail transaction

The present invention is a method and system for measuring viewership of people for a displayed object. The displayed object can be specific in-store marketing elements, such as static signage, POP displays, and other forms of digital media, including retail TV networks and kiosks. In the present invention, the viewership comprises impression level, impression count of the viewers, such as how many people actually viewed said displayed object, average length of impression, distribution of impressions by time of day, and rating of media effectiveness based on audience response. The viewership of people is performed automatically based on the 3-dimensional face pose estimation of the people, using a plurality of means for capturing images and a plurality of computer vision technologies on the captured visual information. The present invention distinguishes viewers from passers-by among the plurality of persons in the vicinity of the displayed object, by counting the number of viewers who actually viewed the displayed object vs. passers-by who may appear in the vicinity of the displayed object but do not actually view the displayed object, using the 3-dimensional face pose estimation and a novel usage of a plurality of computer vision technologies.

Method and system for measuring viewership of people for displayed object

A method and system for analyzing product strength or brand strength by determining shopper decision behavior during a shopping trip. Specifically, shopper behavior can be analyzed to determine whether a shopper's decision to purchase an item occurred at-shelf or pre-shelf. Aggregating decision data across many shoppers over time can then be used to generate analytics regarding the strength of a product or brand. The analysis can then be used to make recommendations to manufacturers or retailers about how to strengthen the product or brand. A deployment of cameras and mobile signal sensors can be utilized to recognize shoppers and track their behavior. Demographics information can also be estimated about the tracked shoppers. The visual and mobile signal trajectories can be fused to form a single shopper trajectory, then associated with Point of Sale (PoS) data. This results in a dataset describing the shopping trip for each tracked shopper.

At-shelf brand strength tracking and decision analytics

A method and system for analyzing shopper switching behavior over multiple shopping trips. Specifically, shopper behavior can be analyzed to generate a shopper behavior profile. Aggregating profile data across many shopping trips over time can then be used to generate analytics regarding the shopper's behavior regarding switching between products or brands and whether the shopper's purchase decisions occurred at-shelf or pre-shelf. Further, the data can be aggregated across many shoppers The analysis can then be used to make recommendations to manufacturers or retailers about how to influence shoppers to switch to a brand from a competitor, or to prevent a shopper from switching to a competitor. A deployment of cameras and mobile signal sensors can be utilized to recognize shoppers and track their behavior. Demographics information can also be estimated about the tracked shoppers. The visual and mobile signal trajectories can be fused to form a single shopper trajectory, then associated with Point of Sale (PoS) data. This results in a dataset describing the shopping trip for each tracked shopper.

Brand-switching analysis using longitudinal tracking of at-shelf shopper behavior

The present invention is a system and method for measuring effectiveness of a marketing campaign on digital signage on many different signage networks, by measuring the efficiency of the campaign at reaching targeted audience and the effectiveness of conveying the message. This invention provides a solution to the challenges created by wide variety of measurements and lack of accuracy. By using automated audience measurement, the current invention is able to collect, large, statistically significant data for analysis. Non-intrusive, computer based measurement also ensure that the data is free from any biases. The media content rating system will provide a quantitative measure of how many people did the campaign reach and what effect did it have. The data will be available at the aggregate level, at network level and down to the screen level.

Method and system for measuring effectiveness of a marketing campaign on digital signage

A method and system for cross-channel shopper behavior analysis. Tracking individual shopper behavior across many retail locations (i.e., across multiple channels) can be extremely valuable for product manufacturers and retails. A configuration of vision sensors and mobile access points can be used to detect and track shoppers as they travel a retail environment, forming a trajectory. The trajectory data can then be associated with Point of Sale data to form a full set of shopper behavior data. The shopper's data can then be aggregated for multiple visits to many retail locations in a geographic area. The aggregated shopper data can be filtered using application-specific criteria for further analysis.

Cross-channel in-store shopper behavior analysis

The present invention provides a comprehensive method for automatically and unobtrusively analyzing the in-store behavior of people visiting a physical space using a multi-modal fusion based on multiple types of sensors. The types of sensors employed may include cameras for capturing a plurality of images and mobile signal sensors for capturing a plurality of Wi-Fi signals. The present invention integrates the plurality of input sensor measurements to reliably and persistently track the people's physical attributes and detect the people's interactions with retail elements. The physical and contextual attributes collected from the processed shopper tracks includes the motion dynamics changes triggered by an implicit and explicit interaction to a retail element, comprising the behavior information for the trip of the people. The present invention integrates point-of-sale transaction data with the shopper behavior by finding and associating the transaction data that corresponds to a shopper trajectory and fusing them to generate a complete an intermediate representation of a shopper trip data, called a TripVector. The shopper behavior analyses are carried out based on the extracted TripVector. The analyzed behavior information for the shopper trips yields exemplary behavior analysis comprising map generation as visualization of the behavior, quantitative shopper metric derivation in multiple scales (e.g., store-wide and category-level) including path-to-purchase shopper metrics (e.g., traffic distribution, shopping action distribution, buying action distribution, conversion funnel), category dynamics (e.g., dominant path, category correlation, category sequence). The present invention includes a set of derived methods for different sensor configurations.

Method and system for in-store shopper behavior analysis with multi-modal sensor fusion

A method and system for associating a physically identifiable feature of a person with the unique identifier of a mobile device carried by the person as detected across repeat visits to a physical location or multiple locations. Specifically, associating the face of a repeat visitor with a unique identifier of a mobile device, such as the device's MAC address, by means of a repeat visit analysis without any explicit or voluntary participation of the person, for example, in a form of providing their information by participation in a survey. For each detection of a particular MAC address associated with a mobile device at a particular physical location, a set of candidate faces can be captured at that physical location. Revisiting faces can be identified and grouped together, and a probability that each group of faces is associated with a particular MAC address can then be calculated. After satisfying several constraints, a maximum probability can be determined, resulting in a match of a particular face to a particular MAC address.

Growing community of inventors

State College, PA, United States of America

Rajeev Sharma