Embedding contextual information in an image to assist understanding

Abstract

A computer-implemented method, system and computer program product for embedding contextual information in an image or video frames. A generative adversarial network (GAN) is trained to provide contextual information to be embedded in an image or video frames, where the contextual information includes text, sound and/or video frames that provides context to the image or video frames. After training the GAN, an image or video frames are received to be embedded with contextual information if necessary. Features are then extracted from the received image/video frames. An image(s) or video frame(s) are identified in a database using the GAN associated with features with a similarity to the extracted features of the received image/video frames that exceeds a threshold value. Such identified images and/or video frames are associated with 'references' containing contextual information which are extracted. The received image/video frames are then augmented with the extracted references to provide context.