Nec Corporation

Nec C Orporation

Renqiang Min

Hans Peter Graf

Eric Cosatto

Asim Kadav

Farley Lai

Dongjin Song

Bing Bai

Alexandru Niculescu-Mizil

Kai Li

Ligong Han

Yunchen Pu

Dinghan Shen

Hui Joyce Zhang

Guofei Jiang

Jianwu Xu

Christopher Malon

Erik Kruus

Xavier Fontaine

Ke Zhang

Yitong Li

Srimat T Chakradhar

Yi Yang

Shaobo Han

Yang Gao

Tingfeng Li

Huayu Li

Linnan Wang

Huahua Wang

Cun Mu

Yizhe Zhu

Yogesh Balaji

Honglu Zhou

Yiren Jian

Pengyu Cheng

A method generates new binding peptides to Major Histocompatibility Complex (MHC) proteins. The method includes training a Generative Adversarial Network (GAN) having a generator and a discriminator only on a set of binding peptide sequences given training data comprising the set of binding peptide sequences and a set of non-binding peptide sequences. A GAN training objective includes the discriminator being iteratively updated to distinguish generated peptide sequences from sampled binding peptide sequences as fake or real and the generator being updated to fool the discriminator. The GAN training objective is optimized while learning two projection vectors for a binding class with two cross-entropy losses. A first loss discriminates binding peptide sequences in the training data from non-binding peptide sequences in the training data. A second loss discriminates generated binding peptide sequences from non-binding peptide sequences in the training data. The method enables supporting decision-making related to new binding peptide generation.

Peptide based vaccine generation system with dual projection generative adversarial networks

A method generates new binding peptides to Major Histocompatibility Complex (MHC) proteins. The method includes training, a Generative Adversarial Network (GAN) having a generator and a discriminator only on a set of binding peptide sequences given training data comprising the set of binding peptide sequences and a set of non-binding peptide sequences. A GAN training objective includes the discriminator being iteratively updated to distinguish generated peptide sequences from sampled binding peptide sequences as fake or real and the generator being updated to fool the discriminator. The GAN training objective is optimized while learning two projection vectors for a binding class with two cross-entropy losses. A first loss discriminates binding peptide sequences in the training data from non-binding peptide sequences in the training data. A second loss discriminates generated binding peptide sequences from non-binding peptide sequences in the training data. The method enables supporting decision-making related to new binding peptide generation.

A method for implementing source-free domain adaptive detection is presented. The method includes, in a pretraining phase, applying strong data augmentation to labeled source images to produce perturbed labeled source images and training an object detection model by using the perturbed labeled source images to generate a source-only model. The method further includes, in an adaptation phase, training a self-trained mean teacher model by generating a weakly augmented image and multiple strongly augmented images from unlabeled target images, generating a plurality of region proposals from the weakly augmented image, selecting a region proposal from the plurality of region proposals as a pseudo ground truth, detecting, by the self-trained mean teacher model, object boxes and selecting pseudo ground truth boxes by employing a confidence constraint and a consistency constraint, and training a student model by using one of the multiple strongly augmented images jointly with an object detection loss.

Source-free cross domain detection method with strong data augmentation and self-trained mean teacher modeling

Systems and methods for predicting T-Cell receptor (TCR)-peptide interaction, including training a deep learning model for the prediction of TCR-peptide interaction by determining a multiple sequence alignment (MSA) for TCR-peptide pair sequences from a dataset of TCR-peptide pair sequences using a sequence analyzer, building TCR structures and peptide structures using the MSA and corresponding structures from a Protein Data Bank (PDB) using a MODELLER, and generating an extended TCR-peptide training dataset based on docking energy scores determined by docking peptides to TCRs using physical modeling based on the TCR structures and peptide structures built using the MODELLER. TCR-peptide pairs are classified and labeled as positive or negative pairs using pseudo-labels based on the docking energy scores, and the deep learning model is iteratively retrained based on the extended TCR-peptide training dataset and the pseudo-labels until convergence.

T-cell receptor repertoire selection prediction with physical model augmented pseudo-labeling for personalized medicine decision making

A method for learning disentangled representations of videos is presented. The method includes feeding each frame of video data into an encoder to produce a sequence of visual features, passing the sequence of visual features through a deep convolutional network to obtain a posterior of a dynamic latent variable and a posterior of a static latent variable, sampling static and dynamic representations from the posterior of the static latent variable and the posterior of the dynamic latent variable, respectively, concatenating the static and dynamic representations to be fed into a decoder to generate reconstructed sequences, and applying three regularizers to the dynamic and static latent variables to trigger representation disentanglement. To facilitate the disentangled sequential representation learning, orthogonal factorization in generative adversarial network (GAN) latent space is leveraged to pre-train a generator as a decoder in the method.

Learning orthogonal factorization in GAN latent space

A reinforcement learning based approach to the problem of query object localization, where an agent is trained to localize objects of interest specified by a small exemplary set. We learn a transferable reward signal formulated using the exemplary set by ordinal metric learning. It enables test-time policy adaptation to new environments where the reward signals are not readily available, and thus outperforms fine-tuning approaches that are limited to annotated images. In addition, the transferable reward allows repurposing of the trained agent for new tasks, such as annotation refinement, or selective localization from multiple common objects across a set of images. Experiments on corrupted MNIST dataset and CU-Birds dataset demonstrate the effectiveness of our approach.

Learning ordinal representations for deep reinforcement learning based object localization

A computer-implemented method is provided for disentangled data generation. The method includes accessing, by a bidirectional Long Short-Term Memory (LSTM) with a multi-head attention mechanism, a dataset including a plurality of pairs each formed from a given one of a plurality of input text structures and given one of a plurality of style labels for the plurality of input text structures. The method further includes training the bidirectional LSTM as an encoder to disentangle a sequential text input into disentangled representations comprising a content embedding and a style embedding based on a subset of the dataset. The method also includes training a unidirectional LSTM as a decoder to generate a next text structure prediction for the sequential text input based on previously generated text structure information and a current word, from a disentangled representation with the content embedding and the style embedding.

Controlled text generation with supervised representation disentanglement and mutual information minimization

Methods and systems for disentangled data generation include accessing a dataset including pairs, each formed from a given input text structure and a given style label for the input text structures. An encoder is trained to disentangle a sequential text input into disentangled representations, including a content embedding and a style embedding, based on a subset of the dataset, using an objective function that includes a regularization term that minimizes mutual information between the content embedding and the style embedding. A generator is trained to generate a text output that includes content from the style embedding, expressed in a style other than that represented by the style embedding of the text input.

Contextual text generation for question answering and text summarization with supervised representation disentanglement and mutual information minimization

A method for using a multi-hop reasoning framework to perform multi-step compositional long-term reasoning is presented. The method includes extracting feature maps and frame-level representations from a video stream by using a convolutional neural network (CNN), performing object representation learning and detection, linking objects through time via tracking to generate object tracks and image feature tracks, feeding the object tracks and the image feature tracks to a multi-hop transformer that hops over frames in the video stream while concurrently attending to one or more of the objects in the video stream until the multi-hop transformer arrives at a correct answer, and employing video representation learning and recognition from the objects and image context to locate a target object within the video stream.

Multi-hop transformer for spatio-temporal reasoning and localization

A computer-implemented method is provided for disentangled data generation. The method includes accessing, by a variational autoencoder, a plurality of supervision signals. The method further includes accessing, by the variational autoencoder, a plurality of auxiliary tasks that utilize the supervision signals as reward signals to learn a disentangled representation. The method also includes training the variational autoencoder to disentangle a sequential data input into a time-invariant factor and a time-varying factor using a self-supervised training approach which is based on outputs of the auxiliary tasks obtained by using the supervision signals to accomplish the plurality of auxiliary tasks.

Self-supervised sequential variational autoencoder for disentangled data generation

A computer-implemented method for employing input-conditioned filters to perform natural language processing tasks using a convolutional neural network architecture includes receiving one or more inputs, generating one or more sets of filters conditioned on respective ones of the one or more inputs by implementing one or more encoders to encode the one or more inputs into one or more respective hidden vectors, and implementing one or more decoders to determine the one or more sets of filters based on the one or more hidden vectors, and performing adaptive convolution by applying the one or more sets of filters to respective ones of the one or more inputs to generate one or more representations.

Convolutional neural network architecture with adaptive filters

Systems and methods for processing video are provided. The method includes receiving a text-based description of active scenes and representing the text-based description as a word embedding matrix. The method includes using a text encoder implemented by neural network to output frame level textual representation and video level representation of the word embedding matrix. The method also includes generating, by a shared generator, frame by frame video based on the frame level textual representation, the video level representation and noise vectors. A frame level and a video level convolutional filter of a video discriminator are generated to classify frames and video of the frame by frame video as true or false. The method also includes training a conditional video generator that includes the text encoder, the video discriminator, and the shared generator in a generative adversarial network to convergence.

Multi-scale text filter conditioned generative adversarial networks

A context-aware attention-based neural network is provided for answering an input question given a set of purportedly supporting statements for the input question. The neural network includes a processing element. The processing element is configured to calculate a question representation for the input question, based on word annotations and word-level attentions calculated for the input question. The processing element is further configured to calculate a sentence representation for each of the purportedly supporting statements, based on word annotations and word-level attentions calculated for each of the purportedly supporting statements. The processing element is also configured to calculate a context representation for the set of purportedly supporting statements with respect to the sentence representation for each of the purportedly supporting statements. The processing element is additionally configured to generate an answer to the input question based on the question representation and the context representation.

Context-aware attention-based neural network for interactive question answering

A method is provided for visual inspection. The method includes learning, by a processor, group disentangled visual feature embedding vectors of input images. The input images include defective objects and defect-free objects. The method further includes generating, by the processor using a weight generation network, classification weights from visual features and semantic descriptions. Both the visual features and the semantic descriptions are for predicting defective and defect-free labels. The method also includes calculating, by the processor, a cosine similarity score between the classification weights and the group disentangled visual feature embedding vectors. The method additionally includes episodically training, by the processor, the weight generation network on the input images to update parameters of the weight generation network. The method further includes generating, by the processor using the trained weight generation network, a prediction of a test image as including any of defective objects and defect-free objects.

Deep group disentangled embedding and network weight generation for visual inspection

A false alarm reduction system and method are provided for reducing false alarms in an automatic defect detection system. The false alarm reduction system includes a defect detection system, generating a list of image boxes marking detected potential defects in an input image. The false alarm reduction system further includes a feature extractor, transforming each of the image boxes in the list into a respective set of numerical features. The false alarm reduction system also includes a classifier, computing as a classification outcome for the each of the image boxes whether the detected potential defect is a true defect or a false alarm responsive to the respective set of numerical features for each of the image boxes.

Growing community of inventors

Princeton, NJ, United States of America

Renqiang Min