Transmutation of virtual entity sketch using extracted features and relationships of real and virtual objects in mixed reality scene

Abstract

A method, system and computer program product for enriching user sketched virtual objects in mixed reality scenes to provide more relevant content. Objects, including a virtual entity sketch and real and virtual objects, are identified in a mixed reality scene. The features for each of these objects are extracted, such as color, texture, parts, etc. A relative position and connectedness of the virtual entity sketch are computed with respect to the mixed reality scene as an object-object matrix using the extracted features. The object-object matrix is then augmented using open and closed regions of the mixed reality scene to create an inter-object relationship graph. The virtual entity sketch is then transmuted using the inter-object relationship graph. Content may then be retrieved using the transmuted virtual entity sketch. In this manner, content that is more relevant to the user can be retrieved based on augmenting and enriching the virtual entity sketch.