Nec Corporation

Nec C Orporation

Nec Hong Kong Limited

Xiang Yu

Manmohan Chandraker

Kihyuk Sohn

Yi-Hsuan Tsai

Bingbing Zhuang

Elsa Wong

Eric Lau

Quoc-Huy Tran

Muhammad Zeeshan Zia

Xi Yin

Chi Li

Donghyun Kim

Masoud Faraki

Yumin Suh

Luan Tran

Samuel Schulter

Feng Zhou

Buyu Liu

Ramin Moslemi

Francesco Pittaluga

Chang Liu

Vijay Kumar Baikampady Gopalkrishna

Xi Peng

Junru Wu

Yichun Shi

Aruni RoyChowdhury

Feng-Ju Chang

Manmohan Chandrakar

Systems and methods are provided for privacy-preserving image feature matching in computer vision applications, including receiving a raw image descriptor, and perturbing the raw image descriptor using a subset selection mechanism to generate a perturbed descriptor set that includes the raw image descriptor and additional descriptors. Each descriptor in the perturbed descriptor set is replaced with its nearest neighbor in a predefined descriptor database to reduce the output domain size of the subset selection mechanism. Local differential privacy (LDP) protocols are employed to further perturb the descriptor set, ensuring formal privacy guarantees, and the perturbed descriptor set is matched against a second set of descriptors for image feature matching.

Image feature matching with formal privacy guarantees

Video methods and systems include extracting features of a first modality and a second modality from a labeled first training dataset in a first domain and an unlabeled second training dataset in a second domain. A video analysis model is trained using contrastive learning on the extracted features, including optimization of a loss function that includes a cross-domain regularization part and a cross-modality regularization part.

Video domain adaptation via contrastive learning for decision making

Video methods and systems include extracting features of a first modality and a second modality from a labeled first training dataset in a first domain and an unlabeled second training dataset in a second domain. A video analysis model is trained using contrastive learning on the extracted features, including optimization of a loss function that includes a cross-domain regularization part and a cross-modality regularization part. The video methods and systems can support real-time decision making in various applications, such as security surveillance where immediate responses to detected actions are required.

Systems and methods are provided for identifying and retrieving semantically similar images from a database. Semantic analysis is performed on an input query utilizing a vision language model to identify semantic concepts associated with the input query. A preliminary set of images is retrieved from the database for semantic concepts identified. Relevant concepts are extracted for images with a tokenizer by comparing images against a predefined label space to identify relevant concepts. A ranked list of relevant concepts is generated based on occurrence frequency within the set. The preliminary set of images is refined based on selecting specific relevant concepts from the ranked list by the user by combining the input query with the specific relevant concepts. Additional semantic analysis is iteratively performed to retrieve additional sets of images semantically similar to the combined input query and selection of the specific relevant concepts until a threshold condition is met.

Multimodal semantic analysis and image retrieval

A method for employing facial information in unsupervised person re-identification is presented. The method includes extracting, by a body feature extractor, body features from a first data stream, extracting, by a head feature extractor, head features from a second data stream, outputting a body descriptor vector from the body feature extractor, outputting a head descriptor vector from the head feature extractor, and concatenating the body descriptor vector and the head descriptor vector to enable a model to generate a descriptor vector.

Face-aware person re-identification system

A method for training a model for face recognition is provided. The method forward trains a training batch of samples to form a face recognition model w(t), and calculates sample weights for the batch. The method obtains a training batch gradient with respect to model weights thereof and updates, using the gradient, the model w(t) to a face recognition model what(t). The method forwards a validation batch of samples to the face recognition model what(t). The method obtains a validation batch gradient, and updates, using the validation batch gradient and what(t), a sample-level importance weight of samples in the training batch to obtain an updated sample-level importance weight. The method obtains a training batch upgraded gradient based on the updated sample-level importance weight of the training batch samples, and updates, using the upgraded gradient, the model w(t) to a trained model w(t+1) corresponding to a next iteration.

Domain generalized margin via meta-learning for deep face recognition

A method for improving face recognition from unseen domains by learning semantically meaningful representations is presented. The method includes obtaining face images with associated identities from a plurality of datasets, randomly selecting two datasets of the plurality of datasets to train a model, sampling batch face images and their corresponding labels, sampling triplet samples including one anchor face image, a sample face image from a same identity, and a sample face image from a different identity than that of the one anchor face image, performing a forward pass by using the samples of the selected two datasets, finding representations of the face images by using a backbone convolutional neural network (CNN), generating covariances from the representations of the face images and the backbone CNN, the covariances made in different spaces by using positive pairs and negative pairs, and employing the covariances to compute a cross-domain similarity loss function.

Face recognition from unseen domains via learning of semantic features

A computer-implemented method for implementing face recognition includes obtaining a face recognition model trained on labeled face data, separating, using a mixture of probability distributions, a plurality of unlabeled faces corresponding to unlabeled face data into a set of one or more overlapping unlabeled faces that include overlapping identities to those in the labeled face data and a set of one or more disjoint unlabeled faces that include disjoint identities to those in the labeled face data, clustering the one or more disjoint unlabeled faces using a graph convolutional network to generate one or more cluster assignments, generating a clustering uncertainty associated with the one or more cluster assignments, and retraining the face recognition model on the labeled face data and the unlabeled face data to improve face recognition performance by incorporating the clustering uncertainty.

Deep face recognition based on clustering over unlabeled face data

A computer-implemented method for implementing face recognition includes receiving training data including a plurality of augmented images each corresponding to a respective one of a plurality of input images augmented by one of a plurality of variations, splitting a feature embedding generated from the training data into a plurality of sub-embeddings each associated with one of the plurality of variations, associating each of the plurality of sub-embeddings with respective ones of a plurality of confidence values, and applying a plurality of losses including a confidence-aware identification loss and a variation-decorrelation loss to the plurality of sub-embeddings and the plurality of confidence values to improve face recognition performance by learning the plurality of sub-embeddings.

Universal feature representation learning for face recognition

An action recognition system is provided that includes a device configured to capture a video sequence formed from a set of unlabeled testing video frames. The system further includes a processor configured to pre-train a recognition engine formed from a reference set of CNNs on a still image domain that includes labeled training still image frames. The processor adapts the recognition engine to a video domain to form an adapted engine, by applying non-reference CNNs to domains that include the still image and video domains and a degraded image domain that includes labeled synthetically degraded versions of the frames in the still image domain. The video domain includes random unlabeled training video frames. The processor recognizes, using the adapted engine, an action performed by at least one object in the sequence, and controls a device to perform a response action in response to an action type of the action.

Action recognition system for action recognition in unlabeled videos with domain adversarial learning and knowledge distillation

A computer-implemented method for implementing face spoofing detection using a physical-cue-guided multi-source multi-channel framework includes receiving a set of data including face recognition data, liveness data and material data associated with at least one face image, obtaining a shared feature from the set of data using a backbone neural network structure, performing, based on the shared feature, a pretext task corresponding to face recognition, a first proxy task corresponding to depth estimation, a liveness detection task, and a second proxy task corresponding to material prediction, and aggregating outputs of the pretext task, the first proxy task, the liveness detection task and the second proxy task using an attention mechanism to boost face spoofing detection performance.

Face spoofing detection using a physical-cue-guided multi-source multi-channel framework

Systems and methods for performing domain adaptation include collecting a labeled source image having a view of an object. Viewpoints of the object in the source image are synthesized to generate view augmented source images. Photometrics of each of the viewpoints of the object are adjusted to generate lighting and view augmented source images. Features are extracted from each of the lighting and view augmented source images with a first feature extractor and from captured images captured by an image capture device with a second feature extractor. The extracted features are classified using domain adaptation with domain adversarial learning between extracted features of the captured images and extracted features of the lighting and view augmented source images. Labeled target images are displayed corresponding to each of the captured images including labels corresponding to classifications of the extracted features of the captured images.

Viewpoint invariant object recognition by synthesization and domain adaptation

A system is provided for pose-variant 3D facial attribute generation. A first stage has a hardware processor based 3D regression network for directly generating a space position map for a 3D shape and a camera perspective matrix from a single input image of a face and further having a rendering layer for rendering a partial texture map of the single input image based on the space position map and the camera perspective matrix. A second stage has a hardware processor based two-part stacked Generative Adversarial Network (GAN) including a Texture Completion GAN (TC-GAN) stacked with a 3D Attribute generation GAN (3DA-GAN). The TC-GAN completes the partial texture map to form a complete texture map based on the partial texture map and the space position map. The 3DA-GAN generates a target facial attribute for the single input image based on the complete texture map and the space position map.

Growing community of inventors

Mountain View, CA, United States of America

Xiang Yu