Microsoft Technology Licensing, LLC

Se Hoon Lim

Timothy Andrew Large

Neil Emerton

Yuqian Zhou

Yonghuan David Ren

Examples are disclosed that relate to the restoration of degraded images acquired via a behind-display camera. One example provides a method of training a machine learning model, the method comprising inputting training image pairs into the machine learning model, each training image pair comprising an undegraded image and a degraded image that represents an appearance of the undegraded image to a behind-display camera, and training the machine learning model using the training image pairs to generate frequency information that is missing from the degraded images.

Image restoration for through-display imaging

Systems and methods for foreground image segmentation, which receive image data captured at a first time, generate a skeleton model of a foreground subject captured in the image data, generate an initial foreground mask for a foreground segment including the foreground subject, generate secondary points corresponding to the foreground subject, identify supplemental points for the foreground segment based on at least the secondary points and the skeleton model, combine the supplemental points with the initial foreground mask to obtain a combined mask, obtain a foreground segment mask for the foreground subject based on the combined mask, and apply the foreground segment mask to the image data to obtain a foreground image portion for the foreground subject.

Skeleton-based supplementation for foreground image segmentation

Techniques for video capture including determining a position of a subject in relation to multiple cameras; selecting a foreground camera from the cameras based on at least the determined position; obtaining an RGB image captured by the foreground camera; segmenting the RGB image to identify a foreground portion corresponding to the subject, with a total height of the foreground portion being a first percentage of a total height of the RGB image; generating a foreground image from the foreground portion; producing a composite image, including compositing the foreground image and a background image to produce a portion of the composite image, with a total height of the foreground image in the composite image being a second percentage of a total height of the composite image and the second percentage being substantially less than the first percentage; and causing the composite image to be displayed on a remote system.

Video capture systems and methods

Techniques for video conferencing including obtaining a first image captured by a camera through a display device of a first device, receiving at the first device live images of a first participant of a video conferencing session, displaying a first composite image on the display device with an image portion of the eyes of the first participant at a lateral position corresponding to the camera, segmenting a foreground image from the first image that corresponds to a second participant of the video conferencing session, and causing via the video conferencing session a second composite image of the second device at a different geographic location, in which the second composite image includes the foreground image composited with a background image.

Growing community of inventors

Bellevue, WA, United States of America

Se Hoon Lim