Vertex Capital LLC

Joan Espadaler Rodes

David Varas Gonzalez

Juan Carlos Riveiro Insua

Asier Aduriz Berasategi

Elisenda Bou Balust

Delia Fernandez Cañellas

An automatic video tagging system which learns from videos, their web context and comments shared on social networks is described. Massive multimedia collections are analyzed by Internet crawling and a knowledge base is maintained that updates in real time with no need of human supervision. As a result, each video is indexed with a rich set of labels and linked with other related contents. Practical applications of video recognition require a label scheme that is appealing to the end-user (i.e. obtained from social curation) and a training dataset that can be updated in real-time to be able to recognize new actions, scenes and people. To create this dataset that evolves in real-time and uses labels that are relevant to the users, a weakly-supervised deep learning approach is utilized combining both a machine-learning pre-processing stage together with a set of keywords obtained from the internet. The resulting tags combined with videos and summaries of videos are used with deep learning to train a neural network in an unsupervised manner that allows the tagging system to go from an image to a set of tags for the image and then to the visual representation of a tag.

Growing community of inventors

Barcelona, Spain

Joan Espadaler Rodes