Microsoft Technology Licensing, LLC

Microsoft Crporation

Hugues Herve Hoppe

John Michael Snyder

Pedro Vieira Sander

Steven Jacob Gortler

Sylvain Lefebvre

Neel Suresh Joshi

Zicheng Liao

John Carlton Platt

Michael F Cohen

Yonatan Wexler

Mark Finch

Matt Uyttendaele

Charles Han

Zoe J Wood

Eyal Ofek

Richard S Szeliski

Sing Bing Kang

Steven Mark Drucker

Darko Kirovski

Kentaro Toyama

Maneesh Agrawala

Georg F Petschnigg

Erin L Renshaw

John W Miller

Alvaro Collet Romea

Baoyuan Wang

Ming Chuang

Matthieu Tony Uyttendaele

Joel Stephen Pritchett

Jing Liao

Diego Fernandes Nehab

Tae-Hyun Oh

Huamin Wang

Adrian Corduneanu

Frank William Losasso Petterson

Steven Jacob Gortier

Michalis Raptis

Chuck Noble

Jovan Popovic

Sisil Sanjeev Mehta

Mark Thomas Finch, Iii

Fabian Andres Prada Nino

Mark Thomas Iii Finch

In some examples, a computing device can determine synthetic meshes based on source meshes of a source mesh sequence and target meshes of a target mesh sequence. The computing device can then place the respective synthetic meshes based at least in part on a rigid transformation to define a processor-generated character. For example, the computing device can determine subsets of the mesh sequences based on a similarity criterion. The computing device can determine modified first and second meshes having a connectivity corresponding to a reference mesh. The computing device can then determine the synthetic meshes based on the modified first and second meshes. In some examples, the computing device can project source and target textures onto the synthetic mesh to provide projected source and target textures. The computing device can determine a synthetic texture registered to the synthetic mesh based on the projected source and target textures.

Motion capture and character synthesis

Various technologies described herein pertain to generating a video loop. An input video can be received, where the input video includes values at pixels over a time range. An optimization can be performed to determine a respective input time interval within the time range of the input video for each pixel from the pixels in the input video. The respective input time interval for a particular pixel can include a per-pixel loop period and a per-pixel start time of a loop at the particular pixel within the time range from the input video. Moreover, an output video can be created based upon the values at the pixels over the respective input time intervals for the pixels in the input video.

Automated video looping with progressive dynamism

Various technologies described herein pertain to generating an output video loop from an input video that includes values at pixels over a time range. Respective input time intervals within the time range of the input video are determined for the pixels by performing an optimization. The optimization can be performed to assign the input time intervals at a first level of resolution, while terms of an objective function use a finer, second level of resolution. An input time interval for a particular pixel includes a per-pixel loop period of a single, contiguous loop at the particular pixel within the time range from the input video. The input time intervals can be temporally scaled based on per-pixel loop periods and an output video loop period. The output video loop is created based on the values at the pixels over the input time intervals for the pixels in the input video.

Video loop generation

A cinemagraph is generated that includes one or more video loops. A cinemagraph generator receives an input video, and semantically segments the frames to identify regions that correspond to semantic objects and the semantic object depicted in each identified region. Input time intervals are then computed for the pixels of the frames of the input video. An input time interval for a particular pixel includes a per-pixel loop period and a per-pixel start time of a loop at the particular pixel. In addition, the input time interval of a pixel is based, in part, on one or more semantic terms which keep pixels associated with the same semantic object in the same video loop. A cinemagraph is then created using the input time intervals computed for the pixels of the frames of the input video.

Generating semantically meaningful video loops in a cinemagraph

Human body motion is represented by a skeletal model derived from image data of a user. The model represents joints and bones and has a rigid body portion. The sets of body data are scaled to a predetermined number of sets for a number of periodic units. A body-based coordinate 3-D reference system having a frame of reference defined with respect to a position within the rigid body portion of the skeletal model is generated. The body-based coordinate 3-D reference system is independent of the camera's field of view. The scaled data and representation of relative motion within an orthogonal body-based 3-D reference system decreases the data and simplifies the calculations for determining motion thus enhancing real-time performance for multimedia applications controlled by a user's natural movements.

Scalable real-time motion recognition

This patent relates to thin plate spline (TPS)-based interpolation techniques for representing free-flowing vector graphics (VG) images based on user-specified features, such as points and curves. One or more features can be identified in a pixel grid. A higher-order least squares interpolating function with a TPS smoothness objective can then be utilized to interpolate individual color values to individual pixels of the pixel grid. Smoothness terms of the function that impose smoothness penalties can be interrupted in certain regions of the pixel grid based on attributes of the user-specified features. For example, a curve attribute can specify a particular color value(s), add or remove a smoothness penalty, or anisotropically impose a first derivative constraint in a particular direction.

Vector graphics with controlled thin-plate splines

Various technologies described herein pertain to juxtaposing still and dynamic imagery to create a cliplet. A first subset of a spatiotemporal volume of pixels in an input video can be set as a static input segment, and the static input segment can be mapped to a background of the cliplet. Further, a second subset of the spatiotemporal volume of pixels in the input video can be set as a dynamic input segment based on a selection of a spatial region, a start time, and an end time within the input video. Moreover, the dynamic input segment can be refined spatially and/or temporally and mapped to an output segment of the cliplet within at least a portion of output frames of the cliplet based on a predefined temporal mapping function, and the output segment can be composited over the background for the output frames of the cliplet.

Juxtaposing still and dynamic imagery for cliplet creation

A dense guide image or signal is used to inform the reconstruction of a target image from a sparse set of target points. The guide image and the set of target points are assumed to be derived from a same real world subject or scene. Potential discontinuities (e.g., tears, edges, gaps, etc.) are first detected in the guide image. The potential discontinuities may be borders of Voronoi regions, perhaps computed using a distance in data space (e.g., color space). The discontinuities and sparse set of points are used to reconstruct the target image. Specifically, pixels of the target image may be interpolated smoothly between neighboring target points, but where neighboring target points are separated by a discontinuity, the interpolation may jump abruptly (e.g., by adjusting or influencing relaxation) at the discontinuity. The target points may be used to select only a subset of the discontinuities to be used during reconstruction.

Reconstruction of sparse data

Embodiments are described for a method to generate an image that includes image structure detail captured from a first image and color from a second image. The first image of a defined subject can be obtained from a computer memory. The first image may be a downsampled fine image with image detail. The second image captured of the defined subject in the first image can be obtained from a computer memory. The second image may be a coarse image. A target pixel in the second image can be selected. A target color distribution for a pixel window of the target pixel can then be computed. A source color distribution for a pixel window of a corresponding pixel in the first image can be computed using a computer processor. Further, a statistic of the target pixel can be determined with respect to the target color distribution. The source color in the source color distribution can be computed with the statistic. The target pixel color can then be replaced by the source color.

Capturing image structure detail from a first image and color from a second image

Embodiments are described for a system and method for generating a multi-resolution image pyramid. The method can include obtaining an image captured as a coarse image of a defined subject and a fine image of the defined subject. The fine image can be downsampled to create a temporary image. A further operation is applying a structure transfer operation to the temporary image to transfer color detail from the coarse image. The structure transfer takes place while retaining structural detail from the temporary image. A blending operation can be applied between the temporary image and the fine image to construct an intermediate image for at least one intermediate level in the multi-resolution image pyramid between the fine image and the coarse image.

Generation of multi-resolution image pyramids

Systems and/or methods that factor large-scale repeated content within and/or among images can include devices and components that factor received or acquired images into epitomes that include all the content of the received or acquired images and transform maps that encode how to construct a facsimile or a close approximation of the image by selecting transformed regions from the epitomes. Through use of both the epitomes and the transform maps in conjunction, a facsimile or a close approximation of the input image can be reconstructed and displayed.

Factoring repeated content within and among images

A method and system for synthesizing texture using upsampled pixel coordinates and a multi-resolution approach. The parallel texture synthesis technique, while based on a neighborhood matching technique having order-independent texture synthesis, extends that approach in at least two areas, including efficient parallel synthesis and intuitive user control. Pixel coordinates are upsampled instead of pixel colors, thereby reducing computational complexity and expense. These upsampled pixel coordinates then are jittered to provide texture variation. The jitter is controllable, such that a user has control over several aspects of the jitter. In addition, each neighborhood-matching pass is split into several sub-passes to improve correction. Using sub-passes improves correction speed and quality. The parallel texture synthesis system and method disclosed herein is designed for implementation on a parallel processor, such as a graphics processing unit.

Parallel texture synthesis by upsampling pixel coordinates

Growing community of inventors

Redmond, WA, United States of America

Hugues Herve Hoppe