Microsoft Technology Licensing, LLC

Nvidia Corporation

Charles T Loop

Zhengyou Zhang

James Frederick Blinn

Towaki Alan Takikawa

Sanja Fidler

Stanley Thomas Birchfield

Jonathan Tremblay

Morgan Samuel McGuire

Karsten Julian Kreis

Stan Birchfield

Kangxue Yin

Joey Litalien

Christian Eisenacher

Jan Kautz

Ying Shan

Nikolai Smolyanskiy

Shalini De Mello

Varun Jampani

Ke Chen

Alexander A Popov

Wonmin Byeon

Mark A B Halstead

Yiran Zhong

Quirin Meyer

Kunal Gupta

Jialiang Wang

Moustafa Meshry

Apparatuses, systems, and techniques are presented to determine distance for one or more objects. In at least one embodiment, a disparity network is trained to determine distance data from input stereoscopic images using a loss function that includes at least one of a gradient loss term and an occlusion loss term.

Distance determinations using one or more neural networks

Apparatuses, systems, and techniques are presented to generate images. In at least one embodiment, ray tracing is caused to be selectively performed on one or more objects within a three-dimensional (3D) environment.

Neural network-based view synthesis

In various examples, methods and systems are provided for estimating depth values for images (e.g., from a monocular sequence). Disclosed approaches may define a search space of potential pixel matches between two images using one or more depth hypothesis planes based at least on a camera pose associated with one or more cameras used to generate the images. A machine learning model(s) may use this search space to predict likelihoods of correspondence between one or more pixels in the images. The predicted likelihoods may be used to compute depth values for one or more of the images. The predicted depth values may be transmitted and used by a machine to perform one or more operations.

Predicting image warping for structure from motion using neural networks

Systems and methods are described for rendering complex surfaces or geometry. In at least one embodiment, neural signed distance functions (SDFs) can be used that efficiently capture multiple levels of detail (LODs), and that can be used to reconstruct multi-dimensional geometry or surfaces with high image quality. An example architecture can represent complex shapes in a compressed format with high visual fidelity, and can generalize across different geometries from a single learned example. Extremely small multi-layer perceptrons (MLPs) can be used with an octree-based feature representation for the learned neural SDFs.

Real-time rendering with implicit shapes

A manifold voxel mesh or surface mesh is manufacturable by carving a single block of material and a non-manifold mesh is not manufacturable. Conventional techniques for constructing or extracting a surface mesh from an input point cloud often produce a non-manifold voxel mesh. Similarly, extracting a surface mesh from a voxel mesh that includes non-manifold geometry produces a surface mesh that includes non-manifold geometry. To ensure that the surface mesh includes only manifold geometry, locations of the non-manifold geometry in the voxel mesh are detected and converted into manifold geometry. The result is a manifold voxel mesh from which a manifold surface mesh of the object may be extracted.

Generation of differentiable, manifold meshes of arbitrary genus

Stereo matching generates a disparity map indicating pixels offsets between matched points in a stereo image pair. A neural network may be used to generate disparity maps in real time by matching image features in stereo images using only 2D convolutions. The proposed method is faster than 3D convolution-based methods, with only a slight accuracy loss and higher generalization capability. A 3D efficient cost aggregation volume is generated by combining cost maps for each disparity level. Different disparity levels correspond to different amounts of shift between pixels in the left and right image pair. In general, each disparity level is inversely proportional to a different distance from the viewpoint.

Neural network system for stereo image matching

Processes and systems described herein enable a computing device to perform voxel-based three-dimensional (3D) reconstruction modeling using parallel graphics hardware. The computing device may generate an initial low resolution voxel list. The voxels of the voxel list may be represented as index triples corresponding to the 3D spatial location of a cell within a voxel grid (e.g., {ix, iy, iz}). Based on information of input images associated with an object, the computing device may remove certain voxels that do not intersect with a surface of the object. The remaining voxels may be subdivided one or more times into a certain number of sub-voxels in parallel to generate a new voxel list. The newly generated voxel list may be tested again to determine voxels that do not intersect with the object surface. This process may repeat in part. The output voxels may be rendered as projected images.

Sparse GPU voxelization for 3D surface reconstruction

A method for rendering parametric surface patches on a display screen includes receiving, at a processing unit, a computer-implemented representation of a first parametric surface patch, wherein the first parametric surface patch is a portion of a three-dimensional computer-implemented model that is desirably displayed at a first viewing perspective on the display screen. The first parametric patch is subdivided in the parameter domain to generate a plurality of subpatches, which are stored as quadtree coordinates in a memory. Thereafter, at least one pixel on the display screen is rendered based at least in part upon the quadtree coordinates in the memory.

Rendering parametric surface patches

Views of parametric surfaces are rendered. A set of parametric surface patches representing a parametric surface being rendered is projected onto a scene, producing a set of view-projected surface patches. Each view-projected surface patch is identified for either culling, subdivision or rendering. For patches which are identified for subdivision, the patches are recursively subdivided into sub-patches until for each sub-patch a prescribed screen-space projection of the sub-patch satisfies a prescribed screen-space error metric. Once the error metric is satisfied, the sub-patch is identified for rendering. Patches and sub-patches which have been identified for rendering are prepared and rendered.

View-dependent rendering of parametric surfaces

A 'Rational Z-Buffer' provides various techniques for reducing artifacts when rendering graphics using z-buffers. In particular, the Rational Z-Buffer reduces the likelihood of z-buffer collisions when using hardware or software z-buffer algorithms to render graphics by delaying homogenous division of pixels until after occlusion testing. Further, occlusion testing between any two pixels, pand p, is accomplished by comparing rational depth values, r={z, w} and r={z, w}, for pixel pand pto determine which pixels are visible. Depth values are compared by determining whether the expression zw&#x3c;zwis true. If true, then pixel poccludes pixel pin clipping cube space relative to an image-plane defined by a particular viewpoint and the field of view. Further, since homogeneous division by w is not performed prior to occlusion testing, rational depth values are uniformly distributed clipping-cube space, thereby significantly reducing potential z collisions relative to conventional z-buffer testing.

Rational Z-buffer for decreasing a likelihood of Z-buffer collisions

Surfaces defined by B&#xe9;zier tetrahedron are generated on programmable graphics hardware. Custom programmed vertex processing, performed by either the CPU or the GPU includes the computation of a symmetric tensor and the assignment of the unique elements of the computed symmetric tensor as vertex attribute data. The vertex attribute data is interpolated by the graphics hardware and output to custom programmed pixel processing. The pixel processing uses the interpolated vertex attribute data to reconstruct, at each pixel, the symmetric tensor which enables the determination of the roots of the polynomial defining the surface to be generated. If no real roots exist, the pixel processing can exit early. If the roots of the polynomial exist, the smallest root can be used as the basis for computing a normal to a point on the surface being rendered, enabling the determination of the color and depth of that pixel.

Real-time GPU rendering of piecewise algebraic surfaces

A shape defined partially be a B&#xe9;zier curve is rendered through a GPU-implemented technique which determines for various screen points which side of the curve the points lie on. This is done in the particular case of cubic B&#xe9;zier curves for shapes defined by the B&#xe9;zier control points of the curves. The type of the curve is identified through an analysis of an inflection point polynomial based on the curve. The curve is then projected to a canonical implicit form in a canonical texture space, allowing computation to be efficiently performed on a simple canonical form of the curve.

Resolution-independent curve rendering using programmable graphics hardware

Surfaces defined by a B&#xe9;zier tetrahedron, and in particular quadric surfaces, are rendered on programmable graphics hardware. Pixels are rendered through triangular sides of the tetrahedra and locations on the shapes, as well as surface normals for lighting evaluations, are computed using pixel shader computations. Additionally, vertex shaders are used to aid interpolation over a small number of values as input to the pixel shaders. Through this, rendering of the surfaces is performed independently of viewing resolution, allowing for advanced level-of-detail management. By individually rendering tetrahedrally-defined surfaces which together form complex shapes, the complex shapes can be rendered in their entirety.

Resolution-independent surface rendering using programmable graphics hardware

Improved triangle management in triangular meshes uses a data structure having two fields to store data for each triangle in the triangular mesh. The first field is a set of three vertices for the triangle and the second field is a set of three edges, each edge corresponding to one of the three vertices. Each of the three edges is an identification of a next or subsequent edge that is encountered when performing a traversal (e.g., in a counterclockwise direction) about the corresponding vertex. According to one aspect, three operators are defined to assist in management of the triangular mesh. These operators are a make edge operator, a splice operator, and a swap operator, and are selectively invoked to both add triangles to the triangular mesh and remove triangles from the triangular mesh.

Growing community of inventors

Seattle, WA, United States of America

Charles T Loop