Microsoft Technology Licensing, LLC

Omnivor, Inc.

Oliver Arthur Whyte

Michael Bleyer

Adam G Kirk

Shahram Izadi

Christoph Rhemann

Sing Bing Kang

Avronil Bhattacharjee

Ross Garrett Cutler

Cha Zhang

Andreas Georgiou

Stanley Thomas Birchfield

Adarsh Prakash Murthy Kowdle

Richard S Szeliski

Charles Lawrence Zitnick, Iii

Emmett Walter Lalish

Amit Mital

Christopher Zach

Carsten Curt Eckard Rother

Adam Garnet Kirk

The subject disclosure is directed towards controlling the intensity of illumination of a scene or part of a scene, including to conserve illumination power. Quality of depth data in stereo images may be measured with different illumination states; environmental conditions, such as ambient light, natural texture may affect the quality. The illumination intensity may be controllably varied to obtain sufficient quality while conserving power. The control may be directed to one or more regions of interest corresponding to an entire scene or part of a scene.

Environment-dependent active illumination for stereo matching

A processor-implemented method of generating a three-dimensional (3D) volumetric video with an overlay representing visibility counts per pixel of a texture atlas, associated with a viewer telemetry data is provided. The method includes (i) capturing the viewer telemetry data, (ii) determining a visibility of each pixel in the texture atlas associated with a 3D content based on the viewer telemetry data, (iii) generating at least one visibility counts per pixel of the texture atlas based on the visibility of each pixel in the texture atlas, and (iv) generating one of: the 3D volumetric video with the overlay of at least one heat map associated with the viewer telemetry data, using the at least one visibility counts per pixel and a curated selection of the 3D volumetric content based on the viewer telemetry data, using the visibility counts per pixel.

Systems and methods for generating a visibility counts per pixel of a texture atlas associated with a viewer telemetry data

Systems and methods for stereo matching based upon active illumination using a patch in a non-actively illuminated image to obtain weights that are used in patch similarity determinations in actively illuminated stereo images is provided. To correlate pixels in actively illuminated stereo images, adaptive support weights computations are used to determine similarity of patches corresponding to the pixels. In order to obtain adaptive support weights for the adaptive support weights computations, weights are obtained by processing a non-actively illuminated ('clean') image.

Active stereo with adaptive support weights from a separate image

A processor implemented method for compressing surface data of a 3 dimensional object in a global digital space, using an image encoder that supports an image data compression algorithm, the image encoder being coupled to a transmitter. The method includes the steps of (i) decomposing the surface data into at least one surface representation that is encoded in an oriented bounding box, (ii) transforming the oriented bounding box into a canonical camera representation to obtain canonical coordinates for the at least one surface representation, (iii) converting each of the at least one surface representation into at least one bounding box image pair that includes a grayscale image representing depth, and a color image and (iv) tiling the at least one bounding box image pair to produce a tiled bounding box image.

System and method for compressing and decompressing surface data of a 3-dimensional object using an image codec

A processor-implemented method for streaming visible blocks of volumetric video to a client device during a predefined time period is provided. The method includes (i) receiving at least one block description file from a content server, (ii) processing each block description in the at least one block description file, to determine the visible blocks that are selected from a set of blocks, that are capable of being visible to a viewer of the client device during the predefined time period, based on a 3D position, size, and an orientation of each block in the set of blocks and at least one view parameter of a user of the client device, (iii) transmitting a request for the visible blocks, to the content server, and (iv) receiving the visible blocks as a visible blocks video at the client device.

Methods for streaming visible blocks of volumetric video

A processor implemented method for compressing time-varying surface data of a 3 dimensional object in a global digital space having frames, using a video encoder that supports a video data compression algorithm, the video encoder being coupled to a transmitter. The method includes the steps of (i) decomposing the time-varying surface data into at least one surface representation that is encoded in an oriented bounding box, (ii) transforming the oriented bounding box into a canonical camera representation for each frame to obtain canonical coordinates for the at least one surface representation, (iii) converting each of the at least one surface representation into at least one bounding box video pair that includes a grayscale video representing depth, and a color video and (iv) tiling the at least one bounding box video pair for each frame to produce a tiled bounding box video.

System and method for compressing and decompressing time-varying surface data of a 3-dimensional object using a video codec

Various examples related to determining a location of an active participant are provided. In one example, image data of a room from an image capture device is received. First audio data from a first microphone array at the image capture device is received. Second audio data from a second microphone array spaced from the image capture device is received. Using a three dimensional model, a location of the second microphone array is determined. Using the first audio data, second audio data, location of the second microphone array, and an angular orientation of the second microphone array, an estimated location of the active participant is determined.

Active speaker location detection

The subject disclosure is directed towards a high resolution, high frame rate, robust stereo depth system. The system provides depth data in varying conditions based upon stereo matching of images, including actively illuminated IR images in some implementations. A clean IR or RGB image may be captured and used with any other captured images in some implementations. Clean IR images may be obtained by using a notch filter to filter out the active illumination pattern. IR stereo cameras, a projector, broad spectrum IR LEDs and one or more other cameras may be incorporated into a single device, which may also include image processing components to internally compute depth data in the device for subsequent output.

Depth imaging system based on stereo vision and infrared radiation

The subject disclosure is directed towards communicating image-related data between a base station and/or one or more satellite computing devices, e.g., tablet computers and/or smartphones. A satellite device captures image data and communicates image-related data (such as the images or depth data processed therefrom) to another device, such as a base station. The receiving device uses the image-related data to enhance depth data (e.g., a depth map) based upon the image data captured from the satellite device, which may be physically closer to something in the scene than the base station, for example. To more accurately capture depth data in various conditions, an active illumination pattern may be projected from the base station or another external projector, whereby satellite units may use the other source's active illumination and thereby need not consume internal power to benefit from active illumination.

Active stereo with satellite device or devices

Various examples related to determining a location of an active speaker are provided. In one example, image data of a room from an image capture device is received and a three dimensional model is generated. First audio data from a first microphone array at the image capture device is received. Second audio data from a second microphone array laterally spaced from the image capture device is received. Using the three dimensional model, a location of the second microphone array with respect to the image capture device is determined. Using the audio data and the location and angular orientation of the second microphone array, an estimated location of the active speaker is determined. Using the estimated location, a setting for the image capture device is determined and outputted to highlight the active speaker.

Growing community of inventors

Seattle, WA, United States of America

Oliver Arthur Whyte