Method and system for scene graph generation

Abstract

The disclosure generally relates to scene graph generation. Scene graph captures rich semantic information of an image by representing objects and their relationships as nodes and edges of a graph and has several applications including image retrieval, action recognition, visual question answering, autonomous driving, robotics. However, to leverage scene graphs, computationally efficient scene graph generation methods are required, which is very challenging to generate due presence of a quadratic number of potential edges and computationally intensive/non-scalable techniques for detecting the relationship between each object pair using the traditional approach. The disclosure proposes a combination of edge proposal neural network and the Graph neural network with spatial message passing (GNN-SMP) along with several techniques including a feature extraction technique, object detection technique, un-labelled graph generation technique and a scene graph generation technique to generate scene graphs.