Microsoft Technology Licensing, LLC

Michael George Boulton

Ashraf A Michail

Jack Elliott

Andrew Zicheng Yeung

Yang You

Robert Warren Gruen

Matthew David Sandy

Brent Michael Wilson

Roberta Rene Moeur

Weige Chen

Bernard Charles Kress

Gerhard A Schneider

Matthew L Bronder

Jingyang Xue

Matt Bronder

Sameer Avinash Nene

Keith D Melmon

Chulian Zhang

Jack Andrew Elliott

Systems are provided for generating training data from images that are obtained from image generators that are typically configured to only generate a single image per frame. The image generators are modified or otherwise controlled to generate two different images at different resolutions for each of a plurality of frames. The training data is created by pairing the low-resolution images and high-resolution images for common frames into training data set pairings. A super-resolution model is applied to the training data set pairings to create a trained super-resolution model.

Generating training data for super resolution models and generating trained super resolution models

A video pass-through computing system includes a head-mounted display device including a display, a camera configured to image a physical scene according to an exposure timing, and an augmented reality control circuit configured to receive a virtual image pixel stream and composite the camera image pixel stream with the virtual image pixel stream to generate a display image pixel stream output to the display, and if a corresponding pixel of the camera image pixel stream is not in temporal synchronization with a pixel of the virtual image pixel stream adjust the exposure timing of the camera.

Video pass-through computing system

An augmented reality display device includes a near-eye display configured to present imagery to a user eye. A camera is configured to capture light from a real-world environment and produce output useable to contribute to the imagery presented to the user eye via the near-eye display. The camera includes an aperture configured to receive the light from the real-world environment and an image sensor configured to respond to the light received from the real-world environment by generating sensor output signals useable to produce images on the near-eye display depicting the real-world environment. One or more optical elements provide an optical path for light from the aperture to the image sensor, the optical path having a length that is within a threshold of a distance between the user eye and the aperture of the camera.

Camera for augmented reality display

A method for efficiently populating a display is provided. The method can include identifying a point at which a world ray intersects a capture surface defined by capture points of a scene, identifying a capture point closest to the identified point, generating a motion vector based on the motion vectors for each of two directly adjacent capture points, identifying a vector in the generated motion vector at a location at which the world ray intersects an image surface, and providing a pixel value from the image data of the capture point, the pixel value corresponding to a location in the image surface at which a vector of the generated motion vector points to the location at which the world ray intersects the image surface within a threshold distance or after a specified number of iterations.

Efficient image population from captured scene

Examples described herein generally relate to displaying an image on a display device where a motion during a latency between a first time associated with activating a first portion of the display device and a second time associated with activating a second portion of the display device is predicted. At least a second portion of an image, to be displayed at the second time, is distorted based at least in part on a function of the motion and the latency to compensate for the latency. A first portion of the image is displayed at the first time by activating the first portion of the display device. The second portion of the image, as distorted, is displayed at the second time by activating the second portion of the display device.

Techniques for compensating variable display device latency in image display

Examples described herein generally relate to performing frame extrapolation in image frame rendering. A vertex mesh as a set of vertices is generated, and each vertex is mapped to a screen space position for defining a texture. One or more motion vectors for one or more regions in a first image frame of a stream of image frames can be determined. The screen space positions associated with at least a portion of the set of vertices within the texture can be modified based at least in part on the one or more motion vectors. A graphics processing unit (GPU) can render the first image frame into the texture. The extrapolated image frame is displayed after the first image frame and before a next image frame in the stream of image frames.

Techniques for extrapolating image frames

Encoding pixel information for pixels of an image. A method includes accessing information defining high-frequency image data correlated with pixels. The method further includes for each pixel, identifying if a vertex from the high-frequency image data is located in that pixel based on analysis of the high-frequency data correlated with the pixel. The method further includes, for one or more pixels in which a vertex is located, identifying the location of the vertex. The method further includes encoding the vertex location into image pixel data.

Vertex pixel buffer

Examples described herein generally relate to rendering graphics in a computing device. A processing over-budget condition related to rendering a frame can be detected, based on which a value of a rendering parameter for a layer, where the layer is one of multiple layers to render for the frame can be modified. The layer can be rendered based at least in part on the value of the rendering parameter while one or more other layers of the multiple layers can be rendered based on respective values for the rendering parameter. The value of the rendering parameter for the layer can be different from at least one of the respective values of the rendering parameter for the one or more other layers.

Techniques for providing dynamic multi-layer rendering in graphics processing

In various embodiments, methods and systems reprojecting three-dimensional (3D) virtual scenes using discontinuity depth late stage reprojection are provided. A reconstruction point, that indicates camera pose information, is accessed. The reconstruction point is associated with a plurality of sample points of a three-dimensional (3D) virtual scene. One or more closest sample points, relative to the reconstruction point, are identified, from the plurality of sample points. Each of the one or more closest sample points is associated with a cube map of color data and depth data. A relative convergence score is determined for each of the one or more closest sample points based on performing a depth-aware cube map late stage reprojection operation in relation to the reconstruction point. A subset of the one or more closest sample points is identified based on the relative convergence score. A reconstructed 3D virtual image is generated using the subset.

Discontinuity-aware reprojection

In various embodiments, methods and systems for reprojecting images based on a velocity depth late stage reprojection process are provided. A reprojection engine supports reprojecting images based on an optimized late stage reprojection process that is performed based on both depth data and velocity data. Image data and corresponding depth and velocity data of the image data is received. A determination of an adjustment to be made to the image is made. The determination is made based on motion data, the depth data and the velocity data. The motion data corresponds to a device associated with displaying the image data. The velocity data supports determining calculated correction distances for portions of the image data. The image data is adjusted based on the determined adjustment. Adjusting the image data is based on integrating depth-data-based translation and velocity-data-based motion correction, into a single pass implementation, to adjust the image data.

Growing community of inventors

Seattle, WA, United States of America

Michael George Boulton