Deep North, Inc.

Seiko Epson Corporation

Nec Laboratories America, Inc.

Other

Sercel Inc.

Jinjun Wang

Jing Xiao

Shun Zhang

Rui Shi

Yudong Liang

Yihong Gong

Xiaomeng Xin

Shenghuo Zhu

Kai Hung Yu

Fengjun Lv

Rohan Sanil

Sanpin Zhou

Zhaowen Wang

Yuanyuan Ding

Wei Xu

Xiaomin Shen

Xi Zhou

Abhijit Deshpande

Michael Zurovec

Andrew Lawrence

Xingyu Wan

Qiqi Hou

A system for image quality assessment of non-aligned images includes a first deep path portion of a convolutional neural network having a set of parameters and a second deep path portion of the convolutional neural network sharing a set of parameters with the first deep path convolutional neural network. Weights are shared between the first and second deep path convolutional neural networks to support extraction of a same set of features in each neural network pathway. Non-aligned reference and distorted images are respectively provided to the first and second deep paths of the convolutional neural network for processing. A concatenation layer is connected to both the first and second deep paths convolutional neural network, and a fully connected layer is connected to the concatenation layer to receive input from both the first and second deep paths of the convolutional neural network, generating an image quality assessment as a linear regressor and outputting an image quality score.

Image quality assessment using similar scenes as reference

Self-checkout verification systems and methods are described. One aspect includes receiving a plurality of images from a camera, the images being associated with a customer self-checkout process. The images may be analyzed to detect one or more items in possession of the customer, count a first number of the items, categorize each item, and construct a first category set including the categorizing for all the items. One aspect includes receiving a point-of-sale record at a completion of the self-checkout process, the point-of-sale record including a second number of the items and a second category set including the categorizing for all the items. The first number and the second number, and the first category set and the second category set may be compared. An alert may be generated if there is a discrepancy between the first number and the second number, or the first category set and the second category set.

Self-checkout verification systems and methods

A semi-supervised model incorporates deep feature learning and pseudo label estimation into a unified framework. The deep feature learning can include multiple convolutional neural networks (CNNs). The CNNs can be trained on available training datasets, tuned using a small amount of labeled training samples, and stored as the original models. Features are then extracted for unlabeled training samples by utilizing the original models. Multi-view clustering is used to cluster features to generate pseudo labels. Then the original models are tuned by using an updated training set that includes labeled training samples and unlabeled training samples with pseudo labels. Iterations of multi-view clustering and tuning using an updated training set can continue until the updated training set is stable.

Semi-supervised person re-identification using multi-view clustering

A facial recognition method using online sparse learning includes initializing target position and scale, extracting positive and negative samples, and extracting high-dimensional Haar-like features. A sparse coding function can be used to determine sparse Haar-like features and form a sparse feature matrix, and the sparse feature matrix in turn is used to classify targets.

Vision based target tracking that distinguishes facial feature targets

A foreground attentive neural network is used to learn feature representations. Discriminative features are extracted from the foreground of the input images. The discriminative features are used for various visual recognition tasks such as person re-identification and multi-target tracking. A deep neural network can include a foreground attentive subnetwork, a body part subnetwork and the feature fusion subnetwork. The foreground attentive subnetwork focuses on foreground by passing each input image through an encoder and decoder network. Then, the encoded feature maps are averagely sliced and discriminately learned in the following body part subnetwork. Afterwards, the resulting feature maps are fused in the feature fusion subnetwork. The final feature vectors are then normalized on the unit sphere space and learned by following the symmetric triplet loss layer.

Foreground attentive feature learning for person re-identification

A multi-object tracking (MOT) framework uses a dual Long Short-Term Memory (LSTM) network (Siamese) for MOT. The dual LSTM network learns metrics along with an online updating scheme for data association. The dual LSTM network fuses relevant features of trajectories to interpret both temporal and spatial components non-linearly and concurrently outputs a similarity score. An LSTM model can be initialized for each trajectory and the metric updated in an online fashion during the tracking phase. An efficient and feasible visual tracking approach using Optical Flow and affine transformations can generate robust tracklets for initialization. Thus, the MOT framework can achieve increased tracking accuracy. Further, the MOT framework has improved performance and can be flexible utilized in arbitrary scenarios.

Multi-object tracking using online metric learning with long short-term memory

A non-hierarchical and iteratively updated tracking system includes a first module for creating an initial trajectory model for multiple targets from a set of received image detections. A second module is connected to the first module to provide identification of multiple targets using a target model, and a third module is connected to the second module to solve a joint object function and maximal condition probability for the target module. A tracklet module can update the first module trajectory module, and after convergence, output a trajectory model for multiple targets.

Vision based target tracking using tracklets

A visual recognition system to process images includes a global sub-network including a convolutional layer and a first max pooling layer. A local sub-network is connected to receive data from the global sub-network, and includes at least two convolutional layers, each connected to a max pooling layer. A fusion network is connected to receive data from the local sub-network, and includes a plurality of fully connected layers that respectively determine local feature maps derived from images. A loss layer is connected to receive data from the fusion network, set filter parameters, and minimize ranking error.

Point to set similarity comparison and deep feature learning for visual recognition

A face replacement system for replacing a target face with a source face can include a facial landmark determination model having a cascade multichannel convolutional neural network (CMC-CNN) to process both the target and the source face. A face warping module is able to warp the source face using determined facial landmarks that match the determined facial landmarks of the target face, and a face selection module is able to select a facial region of interest in the source face. An image blending module is used to blend the target face with the selected source region of interest.

Face replacement and alignment

Growing community of inventors

San Jose, CA, United States of America

Jinjun Wang