Microsoft Technology Licensing, LLC

Avneesh Sud

Pragyana K Mishra

Danyel Aharon Fisher

Lei Zhang

Changhu Wang

Rajeev Prasad

Kancheng Cao

Ayman Malek Abdel Hamid Kaheel

Xinghai Sun

Sumit Amar

Ohil Krishnamurthy Manyam

Aidan C Crook

Xiaoyuan Cui

For each image in a collection of images to be searched, the image is represented as a collection of color-edge words, where each color-edge words includes location information, shape information, and color information. The images may be indexed based on the color-edge words. A user-generated sketch is received as a query and represented as a collection of color-edge words. The collection of color-edge words representing the sketch is compared to the image index to identify search results based on a combination of location similarity, shape similarity, and color similarity.

Color sketch image searching

A system described herein includes a receiver component that receives a tree-structured dataset that includes multiple branches that are hierarchically related to one another. The system also includes an executor component that causes a programmable graphical processing unit to generate a Voronoi treemap based at least in part upon the tree-structured dataset, wherein the Voronoi treemap comprises a plurality of subareas that correspond to the multiple branches, and wherein the Voronoi treemap represents hierarchical relationships between the multiple branches.

Generating Voronoi treemaps

Simplified collaborative searching is provided by pattern recognition such as facial recognition, motion recognition, and the like to provide handsfree functionality. Users join a collaborative search by placing themselves within the field of view of a camera communicationally coupled to a computing device that performs facial recognition and identifies the users, thereby adding them to the collaboration. Users also join by performing simple movements with a portable computing device, such as the ubiquitous mobile phone. A collaboration component tracks the users in the collaboration and identifies them to a search engine, thereby enabling the search engine to perform a collaborative search. The collaboration component also disseminates the collaborative recommendations, either automatically or based upon explicit requests triggered by pattern recognition, including motion recognition and touch recognition. The collaboration component can utilize heuristics to identify individual collaborative sessions, or can utilize existing collaborative infrastructure.

Simplified collaborative searching through pattern recognition

Systems, methods, and computer-readable storage media for web-scale visual search capable of using a combination of visual input modalities are provided. An edgel index is created that includes shape-descriptors, including edgel-based representations, that correspond to each of a plurality of images. Each edgel-based representation includes pixels that depicts edges or boundary contours of an image and is created, at least in part, by segmenting the image into a plurality of image segments and performing a multi-phase contour detection on each segment. Upon receiving a search query having a visual query input, the visual query input is converted into shape-descriptors, including an edgel-based representation, and the shape-descriptors, including the edgel-based representation, of each of the plurality of images is compared with the shape-descriptors, including the edgel-based representation, of the visual query input to identify at least one image of the plurality of images that matches the visual query input.

Visual search using multiple visual input modalities

Growing community of inventors

Kirkland, WA, United States of America

Avneesh Sud