Microsoft Technology Licensing, LLC

Nvidia Corporation

Youding Zhu

Raymond Kirk Price

Michael Bleyer

Denis Demandolx

Drew Edward Steedly

Quentin Simon Charles Miller

Todd Michael Lyon

Denis Claude Pierre Demandolx

Qi Kuan Zhou

David Nister

Neda Cvijetic

Alexandru Octavian Balan

Michael Kroepfl

Ibrahim Eden

Vaibhav Thukral

Ruchi Bhargava

Michael Grabner

Amir Akbarzadeh

Birgit Henke

Yu Sheng

Ivana Stojanovic

Andrew Carley

Enliang Zheng

Jeffrey Liu

Vadim Cugunovs

Jordan Marr

Min Shik Park

An end-to-end system for data generation, map creation using the generated data, and localization to the created map is disclosed. Mapstreams&#x2014;or streams of sensor data, perception outputs from deep neural networks (DNNs), and/or relative trajectory data&#x2014;corresponding to any number of drives by any number of vehicles may be generated and uploaded to the cloud. The mapstreams may be used to generate map data&#x2014;and ultimately a fused high definition (HD) map&#x2014;that represents data generated over a plurality of drives. When localizing to the fused HD map, individual localization results may be generated based on comparisons of real-time data from a sensor modality to map data corresponding to the same sensor modality. This process may be repeated for any number of sensor modalities and the results may be fused together to determine a final fused localization result.

Map creation and localization for autonomous driving applications

Optimizations are provided for generating passthrough visualizations for Head Mounted Displays. The interpupil distance of a user wearing a head-mounted device is determined and a stereo camera pair with a left and right camera is used to capture raw images. The center-line perspectives of the images captured by the left camera have non-parallel alignments with respect to center-line perspectives of any images captured by the right camera. After the raw images are captured, various camera distortion corrections are applied to the images to create corrected images. Epipolar transforms are then applied to the corrected images to create transformed images having parallel center-line perspectives. Thereafter, a depth map is generated of the transformed images. Finally, left and right passthrough visualizations are generated and rendered by reprojecting the transformed left and right images.

IPD correction and reprojection for accurate mixed reality object placement

A virtual reality scene is displayed via a display device. A real-world positioning of a peripheral control device is identified relative to the display device. Video of a real-world scene of a physical environment located behind a display region of the display device is captured via a camera. A real-world portal is selectively displayed via the display device that includes a portion of the real-world scene and simulates a view through the virtual reality scene at a position within the display region that tracks the real-world positioning of the peripheral control device.

Real-world portals for virtual reality displays

A head-mounted display device is disclosed, which includes an at least partially see-through display, a processor configured to detect a physical feature, generate an alignment hologram based on the physical feature, determine a view of the alignment hologram based on a default view matrix for a first eye of a user of the head-mounted display device, display the view of the alignment hologram to the first eye of the user on the at least partially see-through display, output an instruction to the user to enter an adjustment input to visually align the alignment hologram with the physical feature, determine a calibrated view matrix based on the default view matrix and the adjustment input, and adjust a view matrix setting of the head-mounted display device based on the calibrated view matrix.

Alignment based view matrix tuning

Various embodiments relating to using motion based view matrix tuning to calibrate a head-mounted display device are disclosed. In one embodiment, the holograms are rendered with different view matrices, each view matrix corresponding to a different inter-pupillary distance. Upon selection by the user of the most stable hologram, the head-mounted display device can be calibrated to the inter-pupillary distance corresponding to the selected most stable hologram.

Motion based view matrix tuning

Growing community of inventors

Sammamish, WA, United States of America

Youding Zhu