Seiko Epson Corporation

Honeywell International Inc.

Yuanyuan Ding

Jing Xiao

Kenji Fukasawa

Guang Chen

Scott McCloskey

Kwong Wing Au

Jinjun Wang

A robot is made to recognize and manipulate different types of cable harnesses in an assembly line. This is achieved by using a stereo camera system to define a 3D cloud of a given cable harness. Pose information of specific parts of the cable harness are determined from the 3D point cloud, and the cable harness is then re-presented as a collection of primitive geometric shapes of known dimensions, whose positions and orientations follow the spatial position of the represented cable harness. The robot can then manipulate the cable harness by using the simplified representation as a reference.

2D/3D localization and pose estimation of harness cables using a configurable structure representation for robot operations

A stereo camera system produces a stereo image pair of a cable harness, which is used to define a 3D point cloud of the cable harness at its current pose position. Pose information of specific parts of the cable harness are determined from the 3D point cloud, and the cable harness is then re-presented as a collection of primitive geometric shapes of known dimensions, whose positions and orientations follow the spatial position and orientation of the imaged cable harness. The length, position and number of geometric shapes are atomically determined from a 2D image segmentation of one of the images in the stereo image pair.

Geodesic distance based primitive segmentation and fitting for 3D modeling of non-rigid objects from 2D images

A method and system for estimating motion blur of an image associated with a moving object. The direction of one-dimensional motion blur may be estimated by inspecting a power spectrum associated with the image. A radon transform with respect to the image power spectrum is computed in the direction of the motion blur. A family of kernels with respect to the one-dimensional motion blur may then be defined utilizing a shutter triggering sequence associated with an image capturing device. The family of kernels may be modeled utilizing a modulation transfer function (MTF). Each modulation transfer function may be compared with the radon transform of the power spectrum associated with the image via a correlation function. The kernel with highest correlation with respect to the radon transform of the image power spectrum may be employed for de-blurring the image.

Fourier domain blur estimation method and system

Aspects of the present invention comprise generating and using Multi-Order Contextual co-Occurrence (MOCO) descriptors to implicitly model the high level context using detection responses from a baseline object detector. In embodiments, a 1-order context feature is computed as a set of randomized binary comparisons on a response map of the baseline object detector. The statistics of the 1-order binary context features are further calculated to construct a higher-order co-occurrence descriptor, which, in embodiments, may be combined with other features such as the 0-order context features and/or the 1-order features to form the MOCO. In embodiments, combining the MOCO feature with the original image feature, the baseline object detector may be evolved to a stronger context aware detector.

Detector evolution with multi-order contextual co-occurrence

Aspects of the present invention include feature point matching systems and methods. In embodiments, a tree model is used to find candidate matching features for query feature points. In embodiments, the tree model may be pre-learned using a set of sample images, or alternatively, the tree model may be constructed using one or more of the input images. In embodiments, features in one of the stereo images are registered with the tree model, and then features from the other stereo image are queried through the tree model to identify their correspondences in the registered stereo image. As compared to prior brute force matching methodologies, embodiments disclosed herein reduce the complexity and calculation time for determining matching feature points in stereo images.

Tree-model-based stereo matching

Systems and methods for developing and using adaptive threshold values for different input images for object detection are disclosed. In embodiments, detector response histogram-based systems and methods train models for predicting optimal threshold values for different images. In embodiments, when training the model, an optimal threshold value for an image is defined as the value that maximizes the reduction of false positive image patches while preserving as many true positive image patches as possible. Once trained, the model may be used to set different threshold values for different images by inputting a detector response histogram for the image patches of an image into the model to determine a threshold value for detection.

Adaptive threshold for object detection

In a stereoscopic pair of images, global homography at the image level is applied to feature points extracted from connected components (CC) to identify corresponding CC's and feature points, and to discard any CC's that do not have a corresponding pair in the stereoscopic pair of images. Local homography at the CC level is then applied to individual footprint areas of the previously identified paired CC to further clean feature point correspondence. Any CC or feature point or pixel within a paired CC footprint not satisfying local homography constraint is discarded. A correspondence is also extrapolated between unknown pixels within a paired CC footprint using a weighing mechanism and the unknown pixel's surrounding pixels that do have a known correspondence. This provides a dense correspondence of pixels, or feature points, which is then used to create a dense 3D point cloud of identified objects within a 3D space.

Component based correspondence matching for reconstructing cables

Systems and methods for object detection are presented herein. Embodiments of the present invention utilizing a cascade feature, one or more features at different scales, one or more multi-scale features in combination with a perspective feature, or combinations thereof to detect an object of interest in an input image. In embodiments, the various features are used to train classifiers. In embodiments, the trained classifiers are used in detecting an object of interest in one or more input images.

Multi-scale, perspective context, and cascade features for object detection

Embodiments of the present invention include systems and methods for identifying and locating an object in an image. In embodiments, an object in an image may be identified by segmenting a first image of an object into one or more superpixels; extracting local descriptors from the first image, each of the descriptors having an interest point with a location; correlating the local descriptors to the superpixels based on locations of the local descriptors and superpixels; determining a probability for an object label for each of a set of the superpixels; and assigning an object label to each of the set of the superpixels based on the probability and a smoothness factor that includes weighting in terms of one or more of spatial, colors, angular distances between superpixels. The superpixels of an image may be concatenated to predict an object label for the image and to determine the location of the image.

Computer vision methods and systems to recognize and locate an object or objects in one or more images

Aspects of the present invention include object detection training systems and methods and using object detection systems and methods that have been trained. Embodiments presented herein include hybrid learning approaches that combine global classification and local adaptations, which automatically adjust model complexity according to data distribution. Embodiments of the present invention automatically determine model complexity of the local learning algorithm according to the distribution of ambiguous samples. And, embodiments of the local adaptation from global classifier avoid the common under-training problem for local classifier.

Global classifier with local adaption for objection detection

A catadioptric camera having a perspective camera and multiple curved mirrors, images the multiple curved mirrors and uses the epsilon constraint to establish a vertical parallax between points in one mirror and their corresponding reflection in another. An ASIFT transform is applied to all the mirror images to establish a collection of corresponding feature points, and edge detection is applied on mirror images to identify edge pixels. A first edge pixel in a first imaged mirror is selected, its nearest feature points are identified, and a rigid transform is applied to them. The rigid transform is fitted to corresponding feature points in a second imaged mirror. The closest edge pixel to the expected location as determined by the fitted rigid transform is identified, and its distance to the vertical parallax is determined. If the distance is not greater than predefined maximum, then it is deemed correlate to the edge pixel in the first imaged mirror.

Context and epsilon stereo constrained correspondence matching

Systems and methods for object detection that consider background information are presented. Embodiments of the present invention utilizing a feature called Local Difference Pattern (LDP), which is more discriminative for modeling local background image features. In embodiments, the LDP feature is used to train detection models. In embodiments, the LDP feature may be used in detection to differentiate different image background conditions and adaptively adjust classification to yield higher detection rates.

Local difference pattern based local background modeling for object detection

Aspects of the present invention includes systems and methods for generating detection models that consider contextual information of an image patch and for using detection models that consider contextual information. In embodiments, a multi-scale image context descriptor is generated to represent the contextual cues in multiple parameters, such as spatial, scaling, and color spaces. In embodiments, a classification context is defined using the contextual features and is used in a contextual boost classification scheme. In embodiments, the contextual boost propagates contextual cues to larger coverage through iterations to improve the detection accuracy.

Contextual boost for object detection

A content-aware image retargeting technique uses an 'importance filtering' technique to preserve important information in the resizing of an image. The image saliency is first filtered, guided by the image itself to achieve a structure-consistent importance map. The pixel importance is then used as the key constraint in computing the gradient map of pixel shifts from the original resolution to the target resolution. Finally the shift gradient is integrated across the image by a weighted filtering process to construct a smooth pixel shift-map and render the target image. The weight is again controlled by the pixel importance. The two filtering processes enforce the maintaining of structural consistency while preserving the important contents in the target image. The simple nature of the present filter operations allow for real-time applications and easy extension to video retargeting, as the structural constraints from the original image naturally convey the temporal coherence between frames.

Growing community of inventors

Santa Clara, CA, United States of America

Yuanyuan Ding