Interval Research Corporation

Vulcan Patents LLC

Trevor J Darrell

Michael Harville

John Iselin Woodfill

Gaile Gordon

Harlyn Baker

Michele Mae Covell

Ali Rahimi

Michael Hongmai Lin

Segmentation of background and foreground objects in an image is based upon the joint use of range and color data. Range-based data is largely independent of color image data, and hence not adversely affected by the limitations associated with color-based segmentation, such as shadows and similarly colored objects. Furthermore, color segmentation is complementary to range measurement in those cases where reliable range data cannot be obtained. These complementary sets of data are used to provide a multidimensional background estimation. The segmentation of a foreground object in a given frame of an image sequence is carried out by comparing the image frames with background statistics relating to range and normalized color, using the sets of statistics in a complementary manner. A background model is determined by estimating using a multidimensional histogram, recording pixel values, configuring the pixel values into a cluster, and selecting a largest cluster as representing the background model.

Background estimation and segmentation based on range and color

Dense range data obtained at real-time rates is employed to estimate the pose of an articulated figure. In one approach, the range data is used in combination with a model of connected patches. Each patch is the planar convex hull of two circles, and a recursive procedure is carried out to determine an estimate of pose which most closely correlates to the range data. In another aspect of the invention, the dense range data is used in conjunction with image intensity information to improve pose tracking performance. The range information is used to determine the shape of an object, rather than assume a generic model or estimate structure from motion. In this aspect of the invention, a depth constraint equation, which is a counterpart to the classic brightness change constraint equation, is employed. Both constraints are used to jointly solve for motion estimates.

Three dimensional object pose estimation which employs dense depth information

Segmentation of background and foreground objects in an image is based upon the joint use of both range and color data. Range-based data is largely independent of color image data, and hence not adversely affected by the limitations associated with color-based segmentation, such as shadows and similarly colored objects. Furthermore, color segmentation is complementary to range measurement in those cases where reliable range data cannot be obtained. These complementary sets of data are used to provide a multidimensional background estimation. The segmentation of a foreground object in a given frame of an image sequence is carried out by comparing the image frames with background statistics relating to range and normalized color, using the sets of statistics in a complementary manner.

Techniques from computer vision and computer graphics are combined to robustly track a target (e.g., a user) and perform a function based upon the image and/or the identity attributed to the target's face. Three primary modules are used to track a user's head: depth estimation, color segmentation, and pattern classification. The combination of these three techniques allows for robust performance despite unknown background, crowded conditions, and rapidly changing pose or expression of the user. Each of the modules can also provide an identity classification module with valuable information so that the identity of a user can be estimated. With an estimate of the position of a target in 3-D and the target's identity, applications such as individualized computer programs or graphics techniques to distort and/or morph the shape or apparent material properties of the user's face can be performed. The system can track and respond to a user's face in real-time using completely passive and non-invasive techniques.

Method and apparatus for personnel detection and tracking

Image-based synthesis for non-rigid bodies whose appearances do not form a linear manifold is carried out by representing mappings from control parameters to appearances as subsets of piecewise smooth functions. Each subset contains example images which are well approximated by particular examples which lie on the convex hull of the subset's parameter values. Once the subsets of examples are defined, interpolation is performed by using only the examples in a single subset. To provide for efficient operation, image transforms based upon radial cumulative similarities are used to automatically estimate the correspondence between example images.

Example-based image synthesis suitable for articulated figures

A given point of interest in an image is defined by two properties, a local attribute, such as color, and a neighborhood function that describes a similarity pattern. The color value is not influenced by nearby background regions of the image, and functions as a descriptor for each location. The neighborhood function distinguishes locations of similar color from one another, by capturing patterns of change in the local color. The neighborhood function measures the similarity between the local color and colors at nearby points, and reduces the measured similarity values that lie beyond contrast boundaries. Through the computation of such a transform for points of interest in an image, corresponding points in other images can be readily identified.

Growing community of inventors

San Francisco, CA, United States of America

Trevor J Darrell