Microsoft Technology Licensing, LLC

Lu Yuan

Jian Qiao Sun

Jing Liao

Dongdong Chen

Ishani Chakraborty

Zhengyou Zhang

Heung-Yeung Shum

Gang Hua

Thomas M Soemo

Pedro Urbina Escos

Seiichi Kato

Ivan Stojiljkovic

Long Quan

Jonathan C Hanzelka

Jiangyu Liu

Yasufumi Shiraishi

Daniel J Moffitt

Kenji Nikaido

Sing Bing Kang

Fang Wen

Jian Sun

Aleksandar Tomic

Daniel Eduardo Martinez

Yichen Wei

Jifeng Dai

Litian Tao

Yi-Ling Chen

Chintan A Shah

Kaidi Cao

Mingming He

Aleksandar B Tomic

Training a multi-object tracking model includes: generating a plurality of training images based at least on scene generation information, each training image comprising a plurality of objects to be tracked; generating, for each training image, original simulated data based at least on the scene generation information, the original simulated data comprising tag data for a first object; locating, within the original simulated data, tag data for the first object, based on at least an anomaly alert (e.g., occlusion alert, proximity alert, motion alert) associated with the first object in the first training image; based at least on locating the tag data for the first object, modifying at least a portion of the tag data for the first object from the original simulated data, thereby generating preprocessed training data from the original simulated data; and training a multi-object tracking model with the preprocessed training data to produce a trained multi-object tracker.

Training multi-object tracking models using simulation

Image stylization is based on a learning network. A learning network is trained with a plurality of images and a reference image with a particular texture style. A plurality of different sub-networks of the learning network is trained, respectively. Specifically, one of the sub-networks is trained to extract one or more feature maps from the source image and transform the feature maps with the texture style applied thereon to a target image. Each of the feature maps indicates part of feature information of the source image. Another sub-network is trained to apply a specified texture style to the extracted feature maps, such that the target image generated based on the processed feature maps can embody the specified texture style.

Image stylization based on learning network

According to implementations of the subject matter described herein, there is provided an image colorization solution. The solution includes determining a similarity between contents presented in a grayscale source image and a color reference image and determining a col or target image corresponding to the source image based on the similarity. Specifically, a first and a second sets of blocks similar and dissimilar to the reference image are determined based on the similarity; a first color for the first set of blocks is determined based on a color of corresponding blocks in the reference image; a second color for the second set of blocks is determined independently of the reference image. Through this solution, it is possible to provide user controllability and customized effects in colorization, and there is no strict requirement on correspondence between the color image and grayscale image, achieving more robustness to selection of color reference images.

Image colorization based on reference information

Implementations of the present disclosure provide a solution for cross-domain image translation. In this solution, a first learning network for geometric deformation from a first to a second image domain is determined based on first and second images in the first and second domains, images in the two domains having different styles and objects in the images having geometric deformation with respect to each other. Geometric deformation from the second to the first domains is performed on the second image or geometric deformation from the first to the second domains is performed on the first image, to generate an intermediate image. A second learning network for style transfer from the first to the second domains is determined based on the first and intermediate images or based on the second and intermediate images generated. Accordingly, processing accuracy of leaning networks for cross-domain image translation can be improved and complexity is lowered.

Cross-domain image translation

The disclosure herein enables tracking of multiple objects in a real-time video stream. For each individual frame received from the video stream, a frame type of the frame is determined. Based on the individual frame being an object detection frame type, a set of object proposals is detected in the individual frame, associations between the set of object proposals and a set of object tracks are assigned, and statuses of the set of object tracks are updated based on the assigned associations. Based on the individual frame being an object tracking frame type, single-object tracking is performed on the frame based on each object track of the set of object tracks and the set of object tracks is updated based on the performed single-object tracking. For each frame received, a real-time object location data stream is provided based on the set of object tracks.

Tracking multiple objects in a video stream using occlusion-aware single-object tracking

In accordance with implementations of the subject matter described herein, there is proposed a solution of visual stylization of stereoscopic images. In the solution, a first feature map for a first source image and a second feature map for a second source image are extracted. The first and second source images correspond to first and second views of a stereoscopic image, respectively. A first unidirectional disparity from the first source image to the second source image is determined based on the first and second source images. First and second target images having a specified visual style are generated by processing the first and second feature maps based on the first unidirectional disparity. Through the solution, a disparity between two source images of a stereoscopic image are taken into account when performing the visual style transfer, thereby maintaining a stereoscopic effect in the stereoscopic image consisting of the target images.

Visual stylization on stereoscopic images

An 'Adaptive Exposure Corrector' performs automated real-time exposure correction of individual images or image sequences of arbitrary length. 'Exposure correction' is defined herein as automated adjustments or corrections to any combination of shadows, highlights, high-frequency features, and color saturation of images. The Adaptive Exposure Corrector outputs perceptually improved images based on image ISO and camera ISO capabilities in combination with camera noise characteristics via exposure corrections by a variety of noise-aware image processing functions. An initial calibration process adapts these noise aware image processing functions to noise characteristics of particular camera models and types in combination with particular camera ISO settings. More specifically, this calibration process precomputes a Noise Aware Scaling Function (NASF) and a Color Scalar Function (CSF). The NASF and CSF are then applied to adapt various image processing functions that are subsequently applied to perform real-time noise-aware exposure corrections on images as those images are being captured.

Real-time adaptive shadow and highlight enhancement

Various embodiments herein each include at least one of systems, methods, and software for feature flow for video recognition. Such embodiments generally include a fast and accurate framework for video recognition. One example method includes receiving a first frame captured by an imaging device and designating the first frame as a key frame. The method may then generate at least one feature map to identify features in the key frame and subsequently receive a second frame. The method also includes designating the second frame as a current frame and applying a flow estimation algorithm to the key frame and current frame to generate a flow field representing a flow from the key frame to the current frame. The method then propagates each of the at least one feature maps based on the flow field to approximate current locations of features identified within each of the at least one feature maps.

Feature flow for video recognition

Techniques and architectures for video stabilization can transform a shaky video to a steady-looking video. A path smoothing process can generate an optimized camera path for video stabilization. With a large smoothing kernel, a path smoothing process can remove both high frequency jitters and low frequency bounces, and at the same time can preserve discontinuous camera motions (such as quick panning or scene transition) to avoid excessive cropping or geometry distortion. A sliding window based implementation includes a path smoothing process that can be used for real-time video stabilization.

Adaptive path smoothing for video stabilization

Image processing for productivity applications is provided. An image may be received by a computing device. The computing device may detect the edges comprising the received image and adjust the image based on a skew state of the detected edges. The computing device may then process the adjusted image to correct imbalances. The computing device may then assign an image classification to the processed image. The computing device may then adjust the processed image based on the assigned image classification.

Image processing for productivity applications

Embodiments related to the removal of blur from an image are disclosed. One disclosed embodiment provides a method of performing an iterative non-blind deconvolution of a blurred image to form an updated image. The method comprises downsampling the blurred image to form a blurred image pyramid comprising images of two or more different resolution scales, downsampling a blur kernel to form a blur kernel pyramid comprising kernels of two or more different sizes, and deconvoluting a selected image in the blurred image pyramid according to a Richardson-Lucy deconvolution process in which a bilateral range/spatial filter is employed.

Removing blur from an image

A dual-phase approach to red eye correction may prevent overly aggressive or overly conservative red eye reduction. The dual-phase approach may include detecting an eye portion in a digital image. Once the eye portion is detected, the dual-phase approach may include the performance of a strong red eye correction for the eye portion when the eye portion includes a strong red eye. Otherwise, the dual-phase approach may include the performance of a weak red eye correction for the eye portion when the eye portion includes a weak red eye. The weak red eye may be distinguished from the strong red eye based a redness threshold that shows the weak red eye as having less redness hue than the strong red eye.

Growing community of inventors

Beijing, China

Lu Yuan