Fuji Xerox Co., Ltd.

Eastman-kodak Company

Kodak Alaris Inc.

Intellectual Ventures Fund 83 LLC

Dhiraj Joshi

Jiebo Luo

Francine R Chen

Alexander C Loui

Yin-Ying Chen

Jie Yu

Minwoo Park

Lynn D Wilcox

Matthew L Cooper

Peter O Stubler

Madirakshi Das

Charles Austin Parker

Amit Singhal

Phoury Lei

Vivek Kumar Singh

Wei Hao

Huizhong Chen

Bokai Cao

Bor-Chun Chen

Jiebo Lou

Systems and methods to analyze a person's social media photos or videos, such as those posted on Twitter, Facebook, Instagram, etc. and determine properties of their social life. Using information on the number of people appearing in the photos or videos, their ages, and genders, this method can predict whether the person is in a romantic relationship, has a close family, is a group person, or is single. This information is valuable for generating audiovisual content recommendations as well as for advertisers, because it allows targeting personalized advertisements to the person posting the photos. The described methods may be performed (and the advertisements or other content may be selected for recommendation) substantially in real-time as the user accesses a specific online resource.

Systems and methods for gaining knowledge about aspects of social life of a person using visual content associated with that person

A method of generating a caption for a social media post is provided. The method may include receiving a social media post to be posted to a social media network; collecting reference data relevant to determining common activities occurring at a category of location associated with the social media post; inferring potential topics for captions from a topic inference model, based on the collected reference data associated with the social media post; setting parameters associated with a language model based on the inferred topic; and generating at least one caption for the social media post based on the identified language model, and the inferred topic.

Method and apparatus for using business-aware latent topics for image captioning in social media

A method associates social media messages with venues. A social network graph includes users, messages from users, and venues. The venues include multiple primary venues and a no-venue. A link between a message and the no-venue node indicates that the message is not associated with a primary venue. Training feature vectors are constructed that measure connectedness between messages and venues. The process trains a classifier to estimate probabilities that messages are associated with venues. A new social media message is received, and the process constructs a feature vector using the same features as the training vectors, measuring connectedness between the new message and the no-venue. The classifier computes a probability that the new message is associated with the no-venue. When the probability exceeds a predefined threshold, the new message is not associated with any of the primary venues. Otherwise, the new message is associated with one of the primary venues.

Venue link detection for social media messages

Systems and methods disclosed herein associate images with business venues. An example method includes: receiving a first image and retrieving textual reviews and stored images that are associated with one or more candidate business venues. The method further includes: detecting, using trained visual detectors, a plurality of business-aware concepts in the first image and assessing likelihood that detected business-aware concepts are in the first image. The method additionally includes: (i) generating a first representation of the first image based on the likelihoods and one or more term vectors for high-scoring concepts and (ii) receiving second representations of each candidate based on the retrieved textual reviews and stored images. In accordance with determining that the first representation is most similar to a respective second representation of a first candidate, the method includes: (i) associating the first image with the first candidate and (ii) providing an indication of the association.

Systems and methods for associating an image with a business venue by using visually-relevant and business-aware semantics

A method for automatically selecting thematically representative music is disclosed. A processor is used for using a theme-related keyword to search a keyword-indexed video repository to retrieve videos associated with the theme-related keyword; analyzing the retrieved videos to select videos with music; and extracting music tracks and features from the selected videos. The method further includes selecting representative music related to the theme from the extracted music tracks using the extracted features; and storing the selected representative music in a processor accessible memory.

Automatically selecting thematically representative music

A method identifies user demographic characteristics, such as age and gender. First, a plurality of photos are acquired from postings to a social media site by a specific user. A plurality of facial images are identified in the acquired photos. Using a selected subset of the facial images, the method estimates one or more demographic characteristics of the user. In some cases, the estimation uses a profile photograph of the user. In some cases, the facial images are clustered based on similarity. The cluster that most closely matches the profile photo is used for demographic analysis to estimate the characteristics of the user. In some cases, a cluster is selected based on cluster purity, and the images in the cluster are used to estimate the characteristics of the user. In some cases, other self-portrait features are used, such as large facial images or arms along the perimeter of photographs.

Systems and methods for identifying a user's demographic characteristics based on the user's social media photographs

A computer-implemented method performed in connection with a computerized system incorporating a processing unit and a memory, the computer-implemented method involving: using the processing unit to generate a multi-modal language model for co-occurrence of spoken words and displayed text in the plurality of videos; selecting at least a portion of a first video; extracting a plurality of spoken words from the selected portion of the first video; extracting a first displayed text from the selected portion of the first video; and using the processing unit and the generated multi-modal language model to rank the extracted plurality of spoken words based on probability of occurrence conditioned on the extracted first displayed text.

Systems and methods for using latent variable modeling for multi-modal video indexing

A method for viewing a collection of images or videos, includes analyzing the collection to determine properties of the images or videos and using the determined properties to produce icons corresponding to such properties; providing a time-varying display of the images or videos in the collection following an ordering of the images or videos in the collection and at least one of the corresponding icons; receiving a user selection of an icon; and changing the display of the images or videos in the collection following a reordering of the images or videos in the collection in response to the user selection.

Method for media reliving on demand

A method of computing at least one photogenic route from a starting location to a destination location, including; computing photogenic values for images in a large collection representing a geographic region that includes the starting location and the destination location; computing a photogenic index for each route segment based on computed photogenic values of images taken along the route segment; computing at least one photogenic route from the starting location to the destination location and presenting the route(s) to a user.

Generating photogenic routes from starting to destination locations

Generating a tag layout from a set of tags and an ordering of the set of tags, wherein each tag includes a text label and a size for the text label, is disclosed. The system includes a processor accessible memory for receiving an ordered set of tags, each tag including a text label and a size for the text label, and at least one closed shape corresponding to a space for the tag layout. The system further includes a processor for generating the tag layout by computing a scale factor for either the closed shape or the size of the text labels in the set of tags such that all the tags in the set of tags fit within the closed shape, and the processor stores the generated tag layout in the memory.

System for generating tag layouts

A method for viewing a collection of images or videos, includes analyzing the collection to determine properties of the images or videos and using the determined properties to produce icons corresponding to such properties; providing a time-varying display of the images or videos in the collection following an ordering of the images or videos in the collection and at least one of the corresponding icons; receiving a user selection of an icon; changing the time-varying display of the images or videos in the collection following a reordering of the images or videos in the collection in response to the user selection; storing the sequence of the user selections and associated timing in a script in a processor accessible memory; and playing back the viewing of the collection of images or videos using the script.

Method for media reliving playback

Computing a scale factor to insert a first set of shapes into a second set of shapes to form a combined image includes receiving the two sets of shapes, using a processor to convert the first set of shapes into a set of rectangles and the second set of shapes into a set of intervals and computing the scale factor for either the set of intervals or the set of rectangles to generate the combined image by iteratively inserting the set of rectangles into the set of intervals and updating the scale factor in response to a residual area or an overflow area until all the rectangles in the set of rectangles have been inserted into the set of intervals and the residual area in the set of intervals is below a threshold, and storing the combined image in memory.

Method for computing scale for tag insertion

Generating a tag layout from a set of tags and an ordering of the set of tags, wherein each tag includes a text label and a size for the text label, is disclosed. The method further includes receiving at least one closed shape corresponding to a space for the tag layout. A processor computes a scale factor for at least one of the closed shape or the size of the text labels in the set of tags to generate the tag layout of the set of tags within the closed shape such that all the tags in the set of tags fit within the closed shape and the tags are placed in the space based at least upon the ordering of the tags in the set of tags.

Growing community of inventors

Rochester, NY, United States of America

Dhiraj Joshi