Microsoft Technology Licensing, LLC

Google Inc.

Microsoft Corporation

Liyong Chen

Jian Wang

Yingnong Dang

Qiang Wang

Xiaoxu Ma

Jiang Wu

David Robert Gallup

Chase Hensel

Guiheng Zhou

Hao Chen

Jian Jun Wang

Zhouchen Lin

Yu Zou

Shuchang Zhou

Lu Han

Youjun Liu

Shuangfeng Li

Siwei Lyu

Hui Lou

Junjie Ke

Deben Kong

Ze Hong Zhao

Wei Liu

Zheng Chen

Jian Li

Yi Huang

Chunhui Zhang

Steven Maxwell Seitz

Bailiang Zhou

Tanya Matskewich

Mohammed El-Gammal

Ping Hsin Chen

An Xu

David Kilgrow

Jian Qiao

Ming Bai

Yingong Dang

The technology relates to navigating imagery that is organized into clusters based on common patterns exhibited when imagery is captured. For example, a set of captured images which satisfy a predetermined pattern may be determined. The images in the set of set of captured images may be grouped into one or more clusters according to the predetermined pattern. A request to display a first cluster of the one or more clusters may be received and, in response, a first captured image from the requested first cluster may be selected. The selected first captured image may then be displayed.

Cluster based photo navigation

Original data that represents a real-world object or activity and organized along three or more dimensions is received. The original data is represented as a product of several multipliers including a sparse core, such that the sparse core has fewer non-zero values than a tensor representation of the original data, and one or more unitary matrix multipliers. Modified data is generated based on the original data using the multipliers. This includes compressing, or reconstructing missing elements in, the tensor representation of the original data, such that the modified data provides a description of the real-world object or activity that is less complete or more complete, respectively, relative to the original data.

Decomposition techniques for multi-dimensional data

In one aspect, a map is provided for display by one or more computing devices. The map includes a first set of geographic locations. The one or more computing devices receive a zoom request for viewing the map. The one or more computing devices then determine whether a zoom level cutoff has been traversed. If so the one or more computing devices may select a second set of geographic locations, from a hierarchy of geographic locations, to be displayed to the user.

Presenting hierarchies of map data at different zoom levels

Embodiments include a method of generating a three-dimensional (3D) image model using a game-based image acquisition. The method including determining a location of a user device, identifying images to be acquired to generate a 3D model of an area surrounding the location, displaying a virtual gaming view that includes targets corresponding to locations of the images to be acquired, wherein the targets move within the virtual gaming view relative to movement of the user device, and a goal of a game includes moving the user device to align a field of view of a camera of the user device with the targets, in response to determining that the field of view of the camera aligns with a target, acquiring an image depicting an image scene corresponding to the field of view of the camera, and generating a 3D image model of an area surrounding the location using the image.

Systems and methods for generating three-dimensional image models using game-based image acquisition

Methods, systems and computer-readable storage mediums encoded with computer programs executed by one or more processors for providing personalized mapping with photo-tours are disclosed. A mapping system provides a map of a geographic region that includes a plurality of selectable image locations within the region, with standardized images of the respective image location that accord with predefined parameters provided by the mapping system. An image location selection from the map is received and it is determined whether the selected image location is associated with one or more photo-tours of non-standardized images that do not according with the predefined parameters. A selection of a particular one of the photo-tours is received, and the set of non-standardized images corresponding to a selected photo-tour for the image location is provided.

Personalized mapping with photo tours

Aspects of the disclosure relate generally to providing a user with an image navigation experience. In order to do so, a reference image may be identified. A set of potential target images for the reference image may also be identified. A drag vector for user input relative to the reference image is determined. For particular image of the set of target images an associated cost is determined based at least in part on a cost function and the drag vector. A target image is selected based on the determined associated costs.

Techniques for navigation among multiple images

Pre-processing techniques for processing an image to improve the distinctiveness of an information pattern captured in the image before the information pattern is analyzed in a decoding process. The brightness of an image first is normalized by dividing the image into blocks of areas, such as pixels. A brightness distribution value then is determined for each area of the image by fitting the brightness of its surrounding blocks using bilinear interpolation and extrapolation, and a normalized brightness value for each area can then be obtained by dividing the original brightness value by the brightness distribution value. Next, masks are created to distinguish the information pattern from content captured in the image. The masks may be generated based upon contrast differences between the brightness of pixels representing the information pattern, the brightness of pixels representing content, and the brightness of pixels representing the background of the writing medium.

Preprocessing for information pattern analysis

An efficient technique is provided for determining a portion of a document corresponding to a captured image. Areas of a document in which the pattern is at least partially obscured are identified. A reference pixel in the image is selected, and an offset between the pixel and the pattern is determined. A pixel-by-pixel comparison is then made of the image with the document such that the reference pixel is only compared with locations in the document that are both within the identified areas and have the determined offset from the pattern. The comparison with the highest correspondence between the image pixels and the electronic document then identifies the position of the reference pixel relative to the electronic document.

Document portion identification by fast image mapping

Methods and apparatuses that synchronize a paper document to an associated digital document by establishing a mapping. An embedded interactive code (EIC) Document is created as a digital file that serves as an intermediate tier between the paper document and the digital document. Both the paper document and the EIC document are generated while printing the paper document. The EIC document records the corresponding EIC array allocations and a unique document identification number. An image capturing pen may generate a stroke on any page of paper document. With the EIC document, the methods and apparatuses inform an application the page and location on the page of the stroke.

Embedded interaction code document

In accordance with embodiments of the invention, embedded interaction code (EIC) symbols are recognized. EIC dots are generated based on effective EIC symbols, which have been generated by processing an image containing the EIC symbols, by obtaining graylevels of selected positions of the EIC-symbols. Rotated EIC dots are generated based on the EIC dots by determining which grid cells correspond to the EIC symbols and by determining which direction is a correct orientation of the EIC symbols. A homography matrix is updated with orientation information based on the EIC dots. EIC bits are extracted from the rotated EIC dots based on graylevels of selected positions of the rotated EIC dots.

Embedded interaction code recognition

Disclosed embodiments of the invention relate to analyzing document images, which contain positionally encoded information, such as a maze-pattern watermark, and labeling the images based on a degree to which the document's content, such as text, occludes the position-encoding information. Depending on the degree of such occlusion, it may not be possible to extract enough position-encoding bits from a camera-captured image of the document to determine the camera-captured image's location within the document. An analysis-and-labeling module receives, as input, image data output by an image-generation-and-capturing module and off-line training data; performs analysis-and-labeling processing; and outputs image-label information. The results of document-analysis-and-labeling processing may be used for efficiently determining a location of a camera-captured image within a positionally encoded document.

Positionally encoded document image analysis and labeling

A method and apparatus for capturing information encoded within a surface, such as location information or document metadata, and associating the information with a document is described. The captured information may be obtained by a camera associated with a pointing or writing device having a camera, such as an image-capturing pen.

Stroke localization and binding to electronic document

Systems and methods that determine a path of a pen tip as the pen tip is moved across a document are described. The document is watermarked with a maze pattern from which encoded position information is determined. A sequence of images is captured by a camera that is located in a pen. The path of the pen tip is determined by decoding the associated maze pattern and by matching the captured images with document images. If the position coordinates of any frame that is associated with a stroke cannot be determined from m-array decoding, the frames are transformed and then matched with an area of a document image. Once the position coordinates of at least one frame is determined, the position coordinates of other frames are determined by matching the frames in a neighboring area.

Strokes localization by m-array decoding and fast image matching

X-y positions of a digital pen's tip may be determined by using a calibration parameter to map the x-y positions of the respective centers of images captured by the pen's camera. The calibration parameter may be generated by iteratively calculating estimates of the calibration parameter. The iteratively calculated estimates may be based on a gradient of an error function of the iteratively generated estimates. Calibration input data may be produced by a user placing the pen tip in a fixed location on a surface, which may be a positionally encoded medium, such as paper, and then rotating the pen and/or moving the opposite end of the pen in various directions to capture multiple images for use in generating the calibration parameter. A user may perform such a calibration procedure without the need for complicated calibration equipment typically used in connection with conventional calibration techniques.

Growing community of inventors

Beijing, China

Liyong Chen