Nec Laboratories America, Inc.

Nec Corporation

Yuanqing Lin

Xiaoyu Wang

Shenghuo Zhu

Ming M Yang

Kai Hung Yu

Manmohan Chandraker

Silvio Savarese

Shiyu Song

Anelia Angelova

Feng Zhou

Wongun Choi

Fei-Fei Li

Miao Sun

Yu Xiang

Olga Russakovsky

Tianbao Yang

Qi Tian

Yingze Bao

Qi Qian

Will Zou

Saining Xie

Systems and methods are disclosed for deep learning and classifying images of objects by receiving images of objects for training or classification of the objects; producing fine-grained labels of the objects; providing object images to a multi-class convolutional neural network (CNN) having a softmax layer and a final fully connected layer to explicitly model bipartite-graph labels (BGLs); and optimizing the CNN with global back-propagation.

Fine-grained image classification by exploring bipartite-graph labels

A computer-implemented method for detecting objects by using subcategory-aware convolutional neural networks (CNNs) is presented. The method includes generating object region proposals from an image by a region proposal network (RPN) which utilizes subcategory information, and classifying and refining the object region proposals by an object detection network (ODN) that simultaneously performs object category classification, subcategory classification, and bounding box regression. The image is an image pyramid used as input to the RPN and the ODN. The RPN and the ODN each include a feature extrapolating layer to detect object categories with scale variations among the objects.

Subcategory-aware convolutional neural networks for object detection

Systems and methods are disclosed for classifying vehicles by performing scale aware detection; performing detection assisted sampling for convolutional neural network (CNN) training, and performing deep CNN fine-grained image classification to classify the vehicle type.

Object-centric fine-grained image classification

A method to reconstruct 3D model of an object includes receiving with a processor a set of training data including images of the object from various viewpoints; learning a prior comprised of a mean shape describing a commonality of shapes across a category and a set of weighted anchor points encoding similarities between instances in appearance and spatial consistency; matching anchor points across instances to enable learning a mean shape for the category; and modeling the shape of an object instance as a warped version of a category mean, along with instance-specific details.

Semantic dense 3D reconstruction

Methods and systems for distance metric learning include generating two random projection matrices of a dataset from a d-dimensional space into an m-dimensional sub-space, where m is smaller than d. An optimization problem is solved in the m-dimensional subspace to learn a distance metric based on the random projection matrices. The distance metric is recovered in the d-dimensional space.

Efficient distance metric learning for fine-grained visual categorization

An object detector includes a bottom-up object hypotheses generation unit; a top-down object search with supervised descent unit; and an object re-localization unit with a localization model.

Object detection with Regionlets re-localization

Systems and methods are disclosed for detecting an object in an image by determining convolutional neural network responses on the image; mapping the responses back to their spatial locations in the image; and constructing features densely extract shift invariant activations of a convolutional neural network to produce dense features for the image.

Regionlets with shift invariant neural patterns for object detection

Systems and methods are disclosed for autonomous driving with only a single camera by moving object localization in 3D with a real-time framework that harnesses object detection and monocular structure from motion (SFM) through the ground plane estimation; tracking feature points on moving cars a real-time framework to and use the feature points for 3D orientation estimation; and correcting scale drift with ground plane estimation that combines cues from sparse features and dense stereo visual data.

Moving object localization in 3D using a single camera

Systems and methods are disclosed for object detection by receiving an image and extracting features therefrom; applying a learning process to determine sub-regions and select predetermined pooling regions; and performing selective max-pooling to choose one or more feature regions without noises.

Selective max-pooling for object detection

Systems and methods are disclosed for object detection by receiving an image; segmenting the image; extracting features from the image; and performing a dimension-wise spatial layout selection to pick up dimensions inside a discriminative spatial region for classification.

Dimension-wise spatial layout importance selection: an alternative way to handle object deformation

Systems and methods for object detection by receiving an image; segmenting the image and identifying candidate bounding boxes which may contain an object; for each candidate bounding box, dividing the box into overlapped small patches, and extracting dense features from the patches; during a training phase, applying a learning process to learn one or more discriminative classification models to classify negative boxes and positive boxes; and during an operational phase, for a new box generated from the image, applying the learned classification model to classify whether the box contains an object.

Window dependent feature regions and strict spatial layout for object detection

An image retrieval method includes learning multiple object category classifiers with a processor offline and generating classifications scores of images as the semantic attributes; performing vocabulary tree based image retrieval using local features with semantic-aware co-indexing to jointly embed two distinct cues offline for near-duplicate image retrieval; and identifying top similar or dissimilar images using multiple semantic attributes.

Semantic-aware co-indexing for near-duplicate image retrieval

A method for fine-grained image classification on an image includes automatically segmenting one or more objects of interest prior to classification; and combining segmented and original image features before performing final classification.

Image segmentation for large-scale fine-grained recognition

Systems and methods are disclosed to categorize images by detecting local features for each image; applying a tree structure to index local features in the images; and extracting a rank list of candidate images with category tags based on a tree indexing structure to estimate a label of a query image.

Fast image classification by vocabulary tree based image retrieval

A method is provided for classifying an image. The method includes inferring location information of an object of interest in an input representation of the image. The method further includes determining foreground object features and background object features from the input representation of the image. The method additionally includes pooling the foreground object features separately from the background object features using the location information to form a new representation of the image. The new representation is different than the input representation of the image. The method also includes classifying the image based on the new representation of the image.

Growing community of inventors

Sunnyvale, CA, United States of America

Yuanqing Lin