Microsoft Technology Licensing, LLC

William Douglas Guyman

Alexandru Octavian Balan

Drew Edward Steedly

Michael Edward Samples

Ivan Razumenic

Vuk Jovanovic

Andrew Clark Goris

Martin Thomas Shetter

Vladimir Carapic

Taras Khapko

Jelena Mojasevic

Marko Bezulj

Hang Li

Amol Ashok Ambardekar

Szymon Piotr Stachniak

Pedro Urbina Escos

Ning Xu

Houdong Hu

Sheng Kai Tang

Constantin Dulu

Muhammad Zeeshan Zia

Andrey Konin

Vishal Jain

Maria Cheng Lei

Harpreet Singh Sawhney

Salim Sirtkaya

Marc Pollefeys

Michelle Brook

Hendrik Mark Langerak

Filip Panjevic

Julia Gong

Silvano Galliani

Bilha-Catherine W Githinji

Yogeshwar Narayanan Nagaraj

A data processing system implements a receiving a textual input comprising a query for a first image. The data processing system also implements analyzing the textual input to determine a predicted color palette associated with a subject matter of the query; and procedurally generating the first image using the predicted color palette. Another implementation of the data processing system implements providing the textual input to a first machine learning model to obtain the first image, the first machine learning model being trained using a dataset comprising abstract imagery and analyzing the textual input using the first machine learning model to obtain the first image in response to receiving the textual input.

Techniques for abstract image generation from multimodal inputs with content appropriateness considerations

One disclosed example provides a head-mounted device configured to control a plurality of light sources of a handheld object and acquire image data comprising a sequence of environmental tracking exposures in which the plurality of light sources are controlled to have a lower integrated intensity and handheld object tracking exposures in which the plurality of light sources are controlled to have a higher integrated intensity. The instructions are further executable to detect, via an environmental tracking exposure, one or more features of the surrounding environment, determine a pose of the head-mounted device based upon the one or more features of the surrounding environment detected, detect via a handheld object tracking exposure the plurality of light sources of the handheld object, determine a pose of the handheld object relative to the head-mounted device based upon the plurality of light sources detected, and output the pose of the handheld object.

Object and environment tracking via shared sensor

The described implementations relate to images and depth information and generating useful information from the images and depth information. One example can identify planes in a semantically-labeled 3D voxel representation of a scene. The example can infer missing information by extending planes associated with structural elements of the scene. The example can also generate a watertight manifold representation of the scene at least in part from the inferred missing information.

Deriving information from images

A method for object recognition includes, at a computing device, receiving an image of a real-world object. An identity of the real-world object is recognized using an object recognition model trained on a plurality of computer-generated training images. A digital augmentation model corresponding to the real-world object is retrieved, the digital augmentation model including a set of augmentation-specific instructions. A pose of the digital augmentation model is aligned with a pose of the real-world object. An augmentation is provided, the augmentation associated with the real-world object and specified by the augmentation-specific instructions.

Real-world object recognition for computing device

Examples are disclosed that relate to tracking a pose of a handheld object used with a head-mounted display device. In one example, a method comprises: receiving image data from an image sensing system; detecting a plurality of feature points of the handheld object in a frame of the image data; receiving inertial measurement unit (IMU) data from an IMU of the handheld object; based on detecting the plurality of feature points and receiving the IMU data, determining a first pose of the handheld object; determining that at least a portion of the plurality of feature points is not detected in another frame of the image data; using the IMU data, updating the first pose of the handheld object to a second pose; and body-locking the second pose of the handheld object to a body location on a user wearing the head-mounted display device.

Tracking pose of handheld object

One disclosed example provides a computing device configured to receive from an image sensor of a head-mounted device environmental tracking exposures and handheld object tracking exposures, determine a pose of the handheld object with respect to the head-mounted device based upon the handheld object tracking exposures, determine a pose of the head-mounted device with respect to a surrounding environment based upon the environmental tracking exposures, derive a pose of the handheld object relative to the surrounding environment based upon the pose of the handheld object with respect to the head-mounted device and the pose of the head-mounted device with respect to the surrounding environment, and output the pose of the handheld object relative to the surrounding environment for controlling a user interface displayed on the head-mounted device.

Tracking wearable device and handheld object poses

Described are examples for capturing one or more real world images for display with one or more computer-generated images (e.g., holograms). One or more computer-generated images for overlaying over one or more real world images can be received, and a depth for overlaying at least one of the computer-generated images over the one or more real world images can be determined. A lens of a camera can be focused based on this depth, and the one or more real world images can be captured via the camera with the lens focused based on the depth. The one or more real world images can be provided as one or more mixed reality images with the one or more computer-generated images overlaid on the one or more real world images, such that the image is focused on objects near the one or more computer-generated images.

Techniques for setting focus in mixed reality applications

Growing community of inventors

Medina, WA, United States of America

William Douglas Guyman