University of Washington

Shijun Sun

Yongmin Kim

Hyun Wook Park

HyunWook Park

Todd Schoepflin

Lakshmanan Gopishankar

Object segmentation and tracking are improved by including directional information to guide the placement of an active contour (i.e., the elastic curve or 'snake') in estimating the object boundary. In estimating an object boundary the active contour deforms from an initial shape to adjust to image features using an energy minimizing function. The function is guided by external constraint forces and image forces to achieve a minimal total energy of the active contour. Both gradient strength and gradient direction of the image are analyzed in minimizing contour energy for an active contour model.

Video object segmentation using active contour model with directional information

Image data for an image frame are allocated among three groups. In one group are image data which are part of a derived motion boundary, along with image data which differ by at least a threshold amount from a corresponding point among normalized background data. In another group are image data which closely correspond to the normalized background image data. A third group includes the remaining pixels. An initial estimate for the object boundary is adjusted iteratively based on the groupings, followed by application of an active contour model.

Object segmentation with background extraction and moving boundary techniques

A coarsely estimated object boundary is formed by a sequence of edge boundary points. A local affine transformation along the estimated object boundary is identified by analyzing edge energy of a current image frame. A sequence of edge boundary points which have edge energy change ratios exceeding a threshold value is identified as a local affine transformation. A refined estimate of the object boundary is determined for the local affine transformation using a local segmentation process based on a key contour point search strategy. End points of the contour are assumed to be known. An initial key contour point is derived from the coarsely estimate object boundary. Candidate key points then are derived and corresponding curves analyzed to select a curve as the improved estimated boundary for the local affine transformation. The curve having a minimum average edge energy change ratio is selected.

Detecting and segmenting local deformation in a tracked video object

A template is analyzed to determine a step size for searching within a search area. First, the template is padded with pixels to increase its size. Cross-correlation between the padded template and the original template leads to identification of an effective step size. Step sizes for each of horizontal and vertical axes are derived. Windows of the search area, selected based on the step size, then are tested in a fast search stage by correlating the template to the window. Any tested window which has a correlation coefficient exceeding a specific value is a local match. A full search of the vicinity of the local match then is performed for all potential windows within an area bounded by one step to either side of the local match along either axis. The location(s) corresponding to the highest correlation(s) exceeding the threshold value are matches.

Template matching using correlative auto-predicative search

A system for image object tracking and segmentation, includes (i) a modified adaptive resonance theory-2 (M-ART2) model for detecting changes of scenes, (ii) a two-dimensional correlative autopredicitve search (2D CAPS) method for object tracking, (iii) a derivation of contour energy for use in modelling an object edge, and (iv) an active contour model with global relaxation for defining optimal image object boundaries. The modelled object boundary is derived from an edge potential energy calculation and a set of current edge points. For each edge point, several candidate data points are also selected. Rather than calculate an energy value for each potential contours, a solution is achieved by comparing energy differences along a travel path. The selected contour is built in steps. An optimal path is selected from the potential paths.

Video object segmentation using active contour modelling with global relaxation

Adaptive noise filtering is applied to an image frame of HSI data to reduce and more uniformly distribute noise while preserving image feature edges. An adaptive spatial filter includes a plurality of averaging kernels. An appropriate kernel is selected for each pixel for each of the hue and saturation components. A set of thresholds are defined for selecting the kernel for the hue component. Another set of thresholds are defined for selecting the kernel for the saturation component. The kernel for the saturation component is selected by comparing the intensity component to the saturation component thresholds. The kernel for the hue component is selected by comparing the product of intensity component and the saturation component to the hue component thresholds. A color gradient operation is applied to the filtered HSI data to aid in detecting image object boundaries. Object segmentation and other image processing techniques may be performed on the filtered HSI data.

Image processing in HSI color space using adaptive noise filtering

The optical flow of an array of pixels in an image field is determined using adaptive spatial and temporal gradients. Artifacts are avoided for image objects which are moving smoothly relative to the image field background. Data from three image frames are used to determine optical flow. A parameter is defined and determined frame by frame which is used to determine whether to consider the data looking forward from frame k to k&plus;1 or the data looking backward from frame k&minus;1 to frame k in initializing spatial and or temporal gradients for frame k. The parameter identifies signifies the areas of occlusion, so that the gradients looking backward from frame k&minus;1 to frame k can be used for the occluded pixel regions. The gradients looking forward are used in the other areas.

Motion estimation within a sequence of data frames using optical flow with adaptive gradients

Template matching using correlative auto-predictive search

Pixel data forming an image is clustered into groups of data of similar color. Pixels of approximately the same color form one cluster. A vigilance parameter determines how many clusters are derived and how wide a range of colors are included in a cluster. The process for allocating input vectors into clusters is based upon a minimum distance measure to multiple prototype vectors. A given image frame is allocated into clusters in an iterative process. A subset of clusters from an initial image frame are treated as featured clusters. The featured clusters correspond object(s). To track the object, input vectors in subsequent frames are tested to determine whether they correspond to a featured cluster. A scene change is detected when more than a prescribed number of prototype vectors have changed by more than a predetermined amount.

Color clustering for scene change detection and object tracking in video sequences

A template is analyzed to determine step sizes for searching within a search area. The template is analyzed by first padding the template with data points to increase its size. Cross-correlation between the padded template and the original template leads to identification of an effective step size along multiple axes. Step sizes for each of a horizontal, vertical and a third axis are derived. Third axis step sizes may correspond to rotation, scaling factor, subsampling factor, linear distance, time or frequency. Windows of the search area, selected based on the step sizes, then are tested in a fast search by correlating the template to selected windows to derive correlation coefficients. Any tested window which has a correlation coefficient exceeding a given value is a potential match for the template and is subject to a refined stage of comparison.

Growing community of inventors

Seattle, WA, United States of America

Shijun Sun