Microsoft Technology Licensing, LLC

Other

Matthew L Bronder

Michael Alan Dougherty

Joseph C Bertolami

Robert Matthew Craig

John Allen Tardif

Samuel A Mann

Ryan M Geiss

Andrew William Fitzgibbon

Oliver M Williams

Jamie Shotton

Matthew William Lee

Adam Green

Ivan Nevraev

Alexander Nankervis

Michael George Boulton

Jingyang Xue

James Andrew Goossen

Matthew David Sandy

Jack Elliott

Andrew Zicheng Yeung

Sameer Avinash Nene

Keith D Melmon

Hamze M Kalache

Christopher John Tector

Christopher Michael McAllen

Yuqun Cao

Tracey M Frankcom

Chulian Zhang

Jack Andrew Elliott

Jack A Elliott

Systems are provided for generating training data from images that are obtained from image generators that are typically configured to only generate a single image per frame. The image generators are modified or otherwise controlled to generate two different images at different resolutions for each of a plurality of frames. The training data is created by pairing the low-resolution images and high-resolution images for common frames into training data set pairings. A super-resolution model is applied to the training data set pairings to create a trained super-resolution model.

Generating training data for super resolution models and generating trained super resolution models

Examples described herein generally relate to systems and method for streaming a video game at a client device. The client device may transmit video game controls to a streaming server. The client device may receive a video stream encoding video images generated in response to the video game controls from the streaming server. The client device may determine that a video image of the video stream to display in a frame has not been completely received at a designated time prior to display of the frame. The client device may determine an image transformation based on a history of the video images and motion vectors for the video stream. The client device may apply the image transformation to a portion of one or more images corresponding to previous frames. The client device may display a substitute video image in the frame including the portion of the transformed image.

Dead reckoning and latency improvement in 3D game streaming scenario

Enhanced user input devices and user input interfacing systems are provided herein which can reduce perceived interaction latency. In one example, a method of operating a user input interface on a host system includes identifying a target pace for delivery of user input state to an application, and determining, based at least on the target pace, one or more timing parameters for transfer of the user input state from a user input device. The method also includes indicating the one or more timing parameters to the user input device, wherein the user input device responsively transfers the user input state according to the one or more timing parameters.

Latency adjustment of user input devices

Methods and devices for presenting a virtual reality image may include receiving head motion information from a positional tracking system that tracks a position and orientation of a head of a user and receiving virtual motion information independent from the head motion information for a virtual reality simulation. The methods and devices may include generating a first scene of the virtual reality simulation based on the virtual motion information and the head motion information. The methods and devices may also include generating a set of visual cues based on the head motion information and rendering a frame including the set of visual cues and the first scene of the virtual reality simulation. The methods and devices may include transmitting the frame of the virtual reality simulation for presentation on a display device.

Vestibular anchoring

Examples described herein generally relate to prioritizing portions of images for rendering in a computing device. A probability field for prioritizing portions of an image of a scene for processing can be determined, where the probability field includes a set of values each corresponding to a likelihood of a rendering parameter acquiring an altered value between a render time at which at least a portion of the image is rendered and a display time at which the image is displayed. A shaped probability field can be generated based at least in part on applying the probability field to an original target shape associated with a display. The shaped probability field can be provided to a downstream node for prioritizing, based at least in part on one or more of the set of values in the probability field, a portion of the image in processing the image.

Techniques for predictive prioritization of image portions in processing graphics

Example embodiments of the present disclosure provide techniques for receiving measurements from one or more inertial sensors (i.e. accelerometer and angular rate gyros) attached to a device with a camera or other environment capture capability. In one embodiment, the inertial measurements may be combined with pose estimates obtained from computer vision algorithms executing with real time camera images. Using such inertial measurements, a system may more quickly and efficiently obtain higher accuracy orientation estimates of the device with respect to an object known to be stationary in the environment.

Augmented reality computing with inertial sensors

A computer implemented method for synchronizing information from a scene using two heterogeneous sensing devices. Scene capture information is provided by a first sensor and a second sensor. The information comprises video streams including successive frames provided at different frequencies. Each frame is separated by a vertical blanking interval. A video output comprising a stream of successive frames each separated by a vertical blanking interval is rendered based on information in the scene. The method determines whether an adjustment of the first and second video stream relative to the video output stream is required by reference to the video output stream. A correction is then generated to at least one of said vertical blanking intervals.

Heterogeneous image sensor synchronization

Systems and methods for unifying coordinate systems in an augmented reality application or system are disclosed. User devices capture an image of a scene, and determine a location based on the scene image. The scene image may be compared to cartography data or images to determine the location. User devices may propose an origin and orientation or transformation data for a common coordinate system and exchange proposed coordinate system data to agree on a common coordinate system. User devices may also transmit location information to an augmented reality system that then determines an a common coordinate system and transmits coordinate system data such as transformation matrices to the user devices. Images presented to users may be adjusted based on user device locations relative to the coordinate system.

Systems and methods for unifying coordinate systems in augmented reality applications

A method and apparatus for rendering the lighting of virtual objects in an augmented reality display. The method includes determining local and ambient light sources based on data provided by one or more light sensors. The light in the physical lighting environment is accounted for by attributing the light to local light sources and/or ambient light sources. A synthesized physical lighting environment is constructed based on the light characteristics of the local and/or ambient light sources, and is used in properly rendering virtual objects in the augmented reality display.

Augmented lighting environments

The following discloses antialiasing systems and methods. Information about one or more fragments or primitives in a pixel area may be dynamically stored. The stored information may include, for example, depth, color, location and coverage. The coverage and depth information may be tracked at a higher frequency across the pixel than the number of fragments or primitives. Fragments or primitives that enter into a pixel area may be compared with fragments or primitives that have been stored. The comparisons may be based on depth and coverage. Either the incoming fragment or the stored fragment may be deleted based on the comparisons. Information associated with fragments that are preserved may be sampled at any location associated with their coverage area of a pixel. Fragments or primitives that are not discarded may be preserved for a final resolve process, which may incorporate information available from neighboring pixel areas.

Dynamic fragment coverage antialiasing

Systems and methods are disclosed for identifying objects captured by a depth camera by condensing classified image data into centroids of probability that captured objects are correctly identified entities. Output exemplars are processed to detect spatially localized clusters of non-zero probability pixels. For each cluster, a centroid is generated, generally resulting in multiple centroids for each differentiated object. Each centroid may be assigned a confidence value, indicating the likelihood that it corresponds to a true object, based on the size and shape of the cluster, as well as the probabilities of its constituent pixels.

Multiple centroid condensation of probability distribution clouds

Systems and methods are disclosed for generating an image for a user based on an image captured by a scene-facing camera or detector. The user's position relative to a component of the system is determined, and the image captured by the scene-facing detector is modified based on the user's position. The resulting image represents the scene as seen from the perspective of the user. The resulting image may be further modified by augmenting the image with additional images, graphics, or other data.

Growing community of inventors

Redmond, WA, United States of America

Matthew L Bronder