Microsoft Technology Licensing, LLC

Facebook, Inc.

Meta Platforms, Inc.

Krishnan Ramnath

Simon John Baker

Anitha Kannan

Richard S Szeliski

Sudipta Narayan Sinha

Alvaro Collet Romea

Saurabh Mitra

Edward Hsiao

Lakshmie Narayan Viswanathan

Matthew Uyttendaele

Charles Lawrence Zitnick, Iii

Qifa Ke

Hermes Germi Pique Corchs

Dahua Lin

Matthew Adam Simari

Steven Kish

Tullie Murrell

Thomas Ward Meyer

Charles Zitnick

Jiao Li

Juliet Anne Bernstein

Alvarao Collet Romea

Images stored in an information repository are prepared for browsing. For each image in the repository, text in the repository is mined to extract snippets of text about the image which are semantically relevant to the image, and for each of these snippets of text, keyterms are detected in the snippet of text which represent either concepts that are related to the image or entities that are related to the image, and the snippet of text and keyterms are associated with the image. Each keyterm that is associated with each image in the repository is hyperlinked to each other image in the repository that has this keyterm associated therewith. A graphical user interface allows a user to browse the images in the repository by using their associated snippets of text and hyperlinked keyterms.

Browsing images via mined hyperlinked text snippets

A mobile client device is localized based on a captured image by identifying where the client device is located from a set of known locations. The set of known locations is associated with a set of regions, where each region is associated with a set of key frames representing the important features of the region. Latent vectors and keypoints are calculated for each of the key frames and an image captured by the client device. The system compares the latent vectors of the captured image to the latent vectors associated with the regions to determine a subset of similar regions. The system compares the keypoints of the captured image to the keypoints associated with the regions in the subset to determine a best match. This determined location is considered the region of the client device and may be used with other localization information to maintain localization of the client device.

Indexing key frames for localization

In one embodiment, a method includes generating, by a device, first tracking data using a first tracking algorithm, based on first video frames associated with a scene. An augmented-reality (AR) effect may be displayed based on the first tracking data. The device may generate a first confidence score associated with the first tracking data and determine that the first confidence score is above a threshold. The device may generate, based on second video frames subsequent to the first video frames, second tracking data using the first tracking algorithm. The device may determine that an associated second confidence score is below a threshold. In response, the device may generate, based on third video frames subsequent to the second video frames, third tracking data using a second tracking algorithm different from the first. The device may then display the AR effect based on the third tracking data.

Dynamic graceful degradation of augmented-reality effects

In one embodiment, a method includes determining, using one or more location sensors of a computing device, an approximate location of the computing device, identifying a content object located within a threshold distance of the approximate location, wherein an augmented-reality map associates the content object with a stored model of a real-world object and specifies a location of the content object on or relative to the stored model of the real-world object, obtaining an image from a camera of the device, identifying, in the image, a target real-world object that matches the stored model of the real-world object, determining a content object location based on a location of the target real-world object and the location of the content object on or relative to the model of the real-world object, and displaying the content object at the content object location.

Mobile persistent augmented-reality experiences

The claimed subject matter provides for systems and/or methods for identification of instances of an object of interest in 2D images by creating a database of 3D curve models of each desired instance and comparing an image of an object of interest against such 3D curve models of instances. The present application describes identifying and verifying the make and model of a car from a possibly single image&#x2014;after the models have been populated with training data of test images of many makes and models of cars. In one embodiment, an identification system may be constructed by generating a 3D curve model by back-projecting edge points onto a visual hull reconstruction from silhouettes of an instance. The system and methods employ chamfer distance and orientation distance provides reasonable verification performance, as well as an appearance model for the taillights of the car to increase the robustness of the system.

Object identification using 3-D curve matching

The description relates to sentence autocaptioning of images. One example can include a set of information modules and a set of sentence generation modules. The set of information modules can include individual information modules configured to operate on an image or metadata associated with the image to produce image information. The set of sentence generation modules can include individual sentence generation modules configured to operate on the image information to produce a sentence caption for the image.

Autocaptioning of images

An image in a web page may be annotated after deriving information about an image when the image may be displayed on multiple web pages. The web pages that show the image may be analyzed in light of each other to determine metadata about the image, then various additional content may be added to the image. The additional content may be hyperlinks to other webpages. The additional content may be displayed as annotations on top of the images and in other manners. Many embodiments may perform searching, analysis, and classification of images prior to the web page being served.

Image annotations on web pages

Growing community of inventors

Redmond, WA, United States of America

Krishnan Ramnath