Microsoft Technology Licensing, LLC

Uber Technologies, Inc.

Johannes P Kopf

Michael F Cohen

Eric Joel Stollnitz

Sudipta Narayan Sinha

Blaise Aguera Y Arcas

Matthew Uyttendaele

Donald Allen Barnett

Markus Unger

David Maxwell Gedye

Richard S Szeliski

Steven Mark Drucker

Eyal Ofek

Sing Bing Kang

Billy P Chen

Sandeep Kanumuri

Ying-Qing Xu

Geoffrey G Zweig

Wolf Kienzle

Jonathan Paul Rosenberg

Daniel Lischinski

Samuel Thomas Williams

Qing Luan

Timo Pekka Pylvaenaeinen

Don Dongwoo Kim

Casey D Stein

Rick Szeliski

Christopher Stephen Messer

One or more techniques and/or systems are provided for ordering images for panorama stitching and/or for providing a focal point indicator for image capture. For example, one or more images, which may be stitched together to create a panorama of a scene, may be stored within an image stack according to one or more ordering preferences, such as where manually captured images are stored within a first/higher priority region of the image stack as compared to automatically captured images. One or more images within the image stack may be stitched according to a stitching order to create the panorama, such as using images in the first region for a foreground of the panorama. Also, a current position of a camera may be tracked and compared with a focal point of a scene to generate a focal point indicator to assist with capturing a new/current image of the scene.

Image capture and ordering

One or more techniques and/or systems are disclosed for generating improved planar panorama imagery, which may be used to view street-level, human-scale locations along a desired path, for example. One or more desired objects can be detected in source imagery that may be used to produce the resulting planar panorama imagery. Respective object regions comprising a desired object can be identified in one or more frames of the source imagery. One or more portions of the source imagery that do not comprise the object region for the desired object can be joined with at least some of the one or more frames that do comprise the object region for the desired object. The resulting planar panorama imagery can thus comprise the desired object. Also, the planar panorama imagery can be generated so as to lack one or more non-desired objects and/or adverse effects that may result therefrom.

Planar panorama imagery generation

Described is a technology by which an image such as a stitched panorama is automatically cropped based upon predicted quality data with respect to filling missing pixels. The image may be completed, including by completing only those missing pixels that remain after cropping. Predicting quality data may be based on using restricted search spaces corresponding to the missing pixels. The crop is computed based upon the quality data, in which the crop is biased towards including original pixels and excluding predicted low quality pixels. Missing pixels are completed by using restricted search spaces to find replacement values for the missing pixels, and may use histogram matching for texture synthesis.

Image completion including automatic cropping

Among other things, one or more techniques and/or systems are provided for defining transition zones for navigating a visualization. The visualization may be constructed from geometry of a scene and one or more texture images depicted the scene from various viewpoints. A transition zone may correspond to portions of the visualization that do not have a one-to-one correspondence with a single texture image, but are generated from textured geometry (e.g., a projection of texture imagery onto the geometry). Because a translated view may have visual error (e.g., a portion of the translated view is not correctly represented by the textured geometry), one or more transition zones, specifying translated view experiences (e.g., unrestricted view navigation, restricted view navigation, etc.), may be defined. For example, a snapback force may be applied when a current view corresponds to a transition zone having a relatively higher error.

Translated view navigation for visualizations

A document that includes a representation of a two-dimensional (2-D) image may be obtained. A selection indicator indicating a selection of at least a portion of the 2-D image may be obtained. A match correspondence may be determined between the selected portion of the 2-D image and a three-dimensional (3-D) image object stored in an object database, the match correspondence based on a web crawler analysis result. A 3-D rendering of the 3-D image object that corresponds to the selected portion of the 2-D image may be initiated.

Three-dimensional object browsing in documents

A 3D model of an object is rendered using centered images of the object. An algorithm executed locally or in a distributed manner calculates camera positions for the images and determines a virtual camera path based on the camera positions. The application adjusts the images to fit the plane of the virtual camera path and fills in the gaps between the images using transition renderings. To improve user experience, the application also calculates resting positions for navigation stop points using a spring system. Upon constructing the 3D model, the application can transmit the 3D model to a variety of user devices including the network connected device having a camera module that captured the images.

Navigation model to render centered objects using images

Intuitive and user-friendly user interface (UI) techniques are provided for navigating street view applications on a mobile device enabling users to view different angles and segments of available street level images. Additionally, retrieval and presentation of street view images are managed to mitigate delays in retrieval of desired images from a server over wireless connections through techniques such as textual representations, replacement views, scheduling image requests, and comparable ones.

Adjustable and progressive mobile device street view

Techniques and systems are disclosed for navigating human scale image data using aligned perspective images. A consecutive sequence of digital images is stacked together by aligning consecutive images laterally with an image offset between edges of consecutive images corresponding to a distance between respective view windows of the consecutive images. A view window of an image in the sequence is rendered, where the view window of the image corresponds to a desired location. Offset portions of the view window of a desired number of images in the sequence are rendered, for example, alongside the full view of the image at the desired location.

Rendering aligned perspective images

A user interface is presented via which user inputs can be received and maps can be displayed. A user selection of a destination and a user specification of a region of interest on a map are received. The region of interest surrounds the destination on the map. In response to receiving the user specification of the region of interest, a destination map is displayed via the user interface. The destination map includes both the destination and the region of interest, and a layout of one or more roads that include one or more routes to the destination at multiple different scales.

Destination maps user interface

A dynamic wide angle image viewing technique is presented which provides a way to view a wide-angle image while zooming between a wide angle view and a narrower angle view that employs both perspective and non-perspective projection models. In general, this involves first establishing the field of view for a view of the wide angle image that is to be displayed. The view is then rendered and displayed based on the established field of view, such that the projection transitions between a perspective projection associated with narrower angle views and a non-perspective projection (e.g., cylindrical, spherical or some other parameterization) associated with wider-angle views.

Dynamic viewing of wide angle images

Techniques are described for rendering annotations associated with an image. A view of an image maybe shown on a display, and different portions of the image are displayed and undisplayed in the view according to panning and/or zooming of the image within the view. The image may have annotations. An annotation may have a location in the image and may have associated renderable media. The location of the annotation relative to the view may change according to the panning and/or zooming. A strength of the annotation may be computed, the strength changing based the panning and/or zooming of the image. The media may be rendered according to the strength. Whether to render the media may be determined by comparing the strength to a threshold.

Rendering annotations for images

A dynamic tone mapping technique is presented that produces a local tone map for a sub-image of a wide-angle, high dynamic range (HDR), which is used in rendering the sub-image for display. The technique generally involves first computing a global tone map of the wide-angle, HDR image in advance of rendering the sub-image. The global tone map is then used during rendering to compute a local tone map based on the average luminance and contrast of the pixels of the sub-image. In addition, the sub-image can be tone mapped as part of the rendering of a sequence of sub-images during a viewer-executed panning and/or zooming session. In this case, the local tone maps can be kept from changing too rapidly by adding a hysteresis feature to smooth out the intensity changes between successive sub-images.

Viewing wide angle images using dynamic tone mapping

A 'Joint Bilateral Upsampler' uses a high-resolution input signal to guide the interpolation of a low-resolution solution set (derived from a downsampled version of the input signal) from low-to high-resolution. The resulting high-resolution solution set is then saved or applied to the original input signal to produce a high-resolution output signal. The high-resolution solution set is close to what would be produced directly from the input signal without downsampling. However, since the high-resolution solution set is constructed in part from a downsampled version of the input signal, it is computed using significantly less computational overhead and memory than a solution set computed directly from a high-resolution signal. Consequently, the Joint Bilateral Upsampler is advantageous for use in near real-time operations, in applications where user wait times are important, and in systems where computational costs and available memory are limited.

Growing community of inventors

Redmond, WA, United States of America

Johannes P Kopf