Microsoft Technology Licensing, LLC

Intel Corporation

Micorsoft Corporation

Baining Guo

Heung-Yeung Shum

Kun Zhou

Xin Tong

Stephen S Lin

Ying-Qing Xu

Jiaping Wang

Yanyun Chen

Lifeng Wang

John Michael Snyder

Xi Wang

Bo Zhang

Minmin Gong

Sing Bing Kang

Li-Yi Wei

Stephen Ssu-te Lin

Richard S Szeliski

Lin Liang

Xin Huang

Ke Deng

Zhong Ren

Hou Qiming

Weiwei Xu

Lili Cheng

Zhouchen Lin

Changyin Zhou

Sean U Kelly

David P Vronay

Xinguo Liu

Hua Zhong

Xiaohan Shi

Peiran Ren

Xu Yang

Ying Huang

Peter-Pike Johannes Sloan

Zhuowen Tu

Moshe Benezra

Joshua William Buckman

Jinwei Gu

Qiming Hou

Yaohua Hu

Ligang Liu

Eyal Ofek

Zicheng Liu

Tommer Leyvand

Xiaodong Mao

Gang Chen

Szymon Piotr Stachniak

Rui Wang

Shuang Zhao

Yasuyuki Matsushita

Baoyuan Wang

Bruno C Silva

Hongbo Jiang

Yue Dong

Wei Shen

Chen Li

Jin Huang

Lvdi Wang

Jun Wang

Heung Yeung Shum

Jingdan Zhang

Ying Hua Song

Michiel Van De Panne

Baoquan Liu

Jianwei Han

Zhunping Zhang

Yayun Chen

Steve Lin

Bill P Chen

Sun Xin

Kangkang Yin

Yizhou Yu

Bo Sun

Yuanjie Zheng

Alex Pfaffe

Qingshan Zhang

Low-latency gesture detection is described, for example, to compute a gesture class from a live stream of image frames of a user making a gesture, for example, as part of a natural user interface controlling a game system or other system. In examples, machine learning components are trained to learn gesture primitives and at test time, are able to detect gestures using the learned primitives, in a fast, accurate manner. For example, a gesture primitive is a latent (unobserved) variable features of a subset of frames from a sequence of frames depicting a gesture. For example, the subset of frames has many fewer frames than a sequence of frames depicting a complete gesture. In various examples gesture primitives are learnt from instance level features computed by aggregating frame level features to capture temporal structure. In examples frame level features comprise body position and body part articulation state features.

Low-latency gesture detection

Some implementations disclosed herein provide techniques and arrangements to render global light transport in real-time or near real-time. For example, in a pre-computation stage, a first computing device may render points of surfaces (e.g., using multiple light bounces and the like). Attributes for each of the points may be determined. A plurality of machine learning algorithms may be trained using particular attributes from the attributes. For example, a first machine learning algorithm may be trained using a first portion of the attributes and a second machine learning algorithm may be trained using a second portion of the attributes. The trained machine learning algorithms may be used by a second computing device to render components (e.g., diffuse and specular components) of indirect shading in real-time.

Rendering global light transport in real-time using machine learning

Removal of the effects of dust or other impurities on image data is described. In one example, a model of artifact formation from sensor dust is determined. From the model of artifact formation, contextual information in the image and a color consistency constraint may be applied on the dust to remove the dust artifacts. Artifacts may also be removed from multiple images from the same or different cameras or camera settings.

Removal of image artifacts from sensor dust

Described is a technology by which a user interacts with a surface representative of a point cloud data to correct for imperfect scan data. The surface is reconstructed based on the interaction. Real time viewing of the image is facilitated by parallel surface reconstruction. For example, the user may draw strokes to reduce topological ambiguities in poorly-sampled areas. An algorithm automatically adds new oriented sample points to the original point cloud based on the user interaction. Then a new isosurface is generated for the augmented point cloud. The user also may specify the geometry of missing areas of the surface. The user copies a set of points from another point cloud, and places the points around the target area. A new isosurface is then generated.

User-guided surface reconstruction

A video sharing system is described to annotate and navigate tourist videos. An example video sharing system enables non-linear browsing of multiple videos and enriches the browsing experience with contextual and geographic information.

City scene video sharing on digital maps

Described is a search technology in which spatially varying anisotropic reflectance is modeled using image data captured from a single view. Reflectance at each point is represented using a microfacet-based Bidirectional Reflectance Distribution Function (BRDF). Modeling processes the image data, which provides a partial normal distribution function (NDF) for each surface point. The NDF at each selected point is completed by texture synthesis using similar, overlapping partial NDFs from other points. Also described is a scanning device that illuminates a sample surface from a two-dimensional set of light directions using a linear array of LEDs moved over a flat sample.

Modeling anisotropic surface reflectance with microfacet synthesis

Described is a technology by which a ray tracer incorporates a GPU-based kd-tree builder for rendering arbitrary dynamic scenes. For each frame, the ray tracer builds a kd-tree for the scene geometry. The ray tracer spawns and traces eye rays, reflective and refractive rays, and shadow rays. For each ray to be traced, the ray tracer walks through the kd-tree until it reaches leaf nodes and associated triangles. When a ray passes through both sides of a splitting plane, the 'far' sub-tree is pushed into the stack and the 'near' sub-tree is traversed first.

Ray tracing on graphics hardware using kd-trees

A system for reflectance acquisition of a target includes a light source, an image capture device, and a reflectance reference chart. The reflectance reference chart is fixed relative to the target. The light source provides a uniform band of light across at least a dimension of the target. The image capture device is configured and positioned to encompass at least a portion of the target and at least a portion of the reflectance reference chart within a field-of-view of the image capture device. The image capture device captures a sequence of images of the target and the reflectance reference chart during a scan thereof. Reflectance responses are calculated for the pixels in the sequence of images. Reference reflectance response distribution functions are matched to the calculated reflectance responses, and an image of the target is reconstructed based at least in part on the matched reference reflectance response distribution functions.

Obtaining spatially varying bidirectional reflectance distribution function

Described is a technology by which a GPU-based photon mapping mechanism/algorithm uses a kd-tree to render arbitrary dynamic scenes. For each frame, the mechanism emits and traces a set of photons into the scene. When a photon hits a surface, it can either be reflected, transmitted, or absorbed based on the surface material. Once photon tracing is done, a kd-tree is built for the stored photons. To estimate the radiance value at an arbitrary surface point, the k-nearest photons are located and filtered. The photon tracing and photon kd-tree construction, as well as the radiance estimation using k-nearest neighbor (KNN) searches are performed on graphics hardware, e.g., a GPU. In one example, only caustic photons are traced, whereby a photon is terminated and stored once it hits a diffuse surface.

Photon mapping on graphics hardware using kd-trees

A light gathering process may reduce the computational resources and storage required to render a scene with a participating homogeneous media. According to some implementations, Efficiency may be obtained by evaluating the final radiance along a viewing ray directly from the lighting rays passing near to it, and by rapidly identifying such lighting rays in the scene. To facilitate a search for nearby lighting rays, the lighting rays and viewing rays may be represented as a 6D point and a plane according to the corresponding Plucker coordinates and coefficients, respectively.

Line space gathering for single scattering in large scenes

Described is a technology in a computing environment comprising a programming language for general purpose computation on a graphics processing unit (GPU), along with an associated compiler. A Bulk-Synchronous GPU Programming (BSGP) program is programmed to include barriers to describe parallel processing on GPUs. A BSGP compiler detects barriers corresponding to supersteps, converts BSGP programs to kernels based on the barriers, and combines them. During compilation, the compiler aligns barriers in the statements and bundles the corresponding supersteps together. A par construct is provided to allow the programmer to control aspects of bundling, e.g., by specifying a block independent statements. Thread manipulation emulation is provided to transparently emulate thread creation and destruction, with operations fork and kill. Also provided is remote variable access intrinsics for efficient communications between threads, and collective primitive operations.

Bulk-synchronous graphics processing unit programming

A computer implemented method for deforming a 3D polygon mesh using non-linear and linear constraints. The method includes creating a coarse control 3D polygon mesh that completely encapsulates the 3D polygon mesh to be deformed, projecting the deformation energy of the 3D polygon mesh and the constraints of the 3D polygon mesh to the vertices, or subspace, of the coarse control 3D polygon mesh, and determining the resulting deformed 3D polygon mesh by iteratively determining the deformation energy of the subspace. The constraints may be either linear or non-linear constraints, for example, a Laplacian constraint, a position constraint, a projection constraint, a skeleton constraint, or a volume constraint.

Three dimensional polygon mesh deformation using subspace energy projection

An exemplary method includes providing image data for an illuminated physical sample of a heterogeneous translucent material, determining one or more material properties of the material based in part on a diffusion equation where one of the material properties is a diffusion coefficient for diffusion of radiation in the material and where the determining includes a regularization term for the diffusion coefficient, mapping the one or more material properties to a virtual object volume, assigning virtual illumination conditions to the virtual object volume, and rendering the virtual object volume using the virtual illumination conditions as a boundary condition for a system of diffusion equations of the virtual object volume. Other methods, devices and systems are also disclosed.

Modeling and rendering of heterogeneous translucent materals using the diffusion equation

Growing community of inventors

Beijing, China

Baining Guo