Nec Corporation

Xuchao Zhang

Haifeng Chen

Bo Zong

Yanchi Liu

Wei Cheng

Yuncong Chen

Zhengzhang Chen

LuAn Tang

Cristian Lumezanu

Dongjin Song

Wenchao Yu

Takehiko Mizoguchi

Shengming Zhang

Zexue He

Junxiang Wang

A method for implementing a self-attentive encoder-decoder transformer framework for anomaly detection in event sequences is presented. The method includes feeding event content information into a content-awareness layer to generate event representations, inputting, into an encoder, event sequences of two hierarchies to capture long-term and short-term patterns and to generate feature maps, adding, in the decoder, a special sequence token at a beginning of an input sequence under detection, during a training stage, applying a one-class objective to bound the decoded special sequence token with a reconstruction loss for sequence forecasting using the generated feature maps from the encoder, and during a testing stage, labeling any event representation whose decoded special sequence token lies outside a hypersphere as an anomaly.

Efficient transformer for content-aware anomaly detection in event sequences

A computer-implemented method for multi-model representation learning is provided. The method includes encoding, by a trained time series (TS) encoder, an input TS segment into a TS-shared latent representation and a TS-private latent representation. The method further includes generating, by a trained text generator, a natural language text that explains the input TS segment, responsive to the TS-shared latent representation, the TS-private latent representation, and a text-private latent representation.

Co-disentagled series/text multi-modal representation learning for controllable generation

A method for detecting business intent from a business intent corpus by employing an Intent Detection via Multi-hop Unified Syntactic Graph (IDMG) is presented. The method includes parsing each text sample representing a business need description to extract syntactic information including at least tokens and words, tokenizing the words of the syntactic information to generate sub-words for each of the words by employing a multi-lingual pre-trained language model, aligning the generated sub-words to the tokens of the syntactic information to match ground-truth intent actions and objects to the tokenized sub-words, generating a unified syntactic graph, encoding, via a multi-hop unified syntactic graph encoder, the unified syntactic graph to generate an output, and predicting an intent action and object from the output.

Intent detection via multi-hop unified syntactic graph

Systems and methods are provided for Cross-lingual Transfer Interpretation (CTI). The method includes receiving text corpus data including premise-hypothesis pairs with a relationship label in a source language, and conducting a source to target language translation. The method further includes performing a feature importance extraction, where an integrated gradient is applied to assign an importance score to each input feature, and performing a cross-lingual feature alignment, where tokens in the source language are aligned with tokens in the target language for both the premise and the hypothesis based on semantic similarity. The method further includes performing a qualitative analysis, where the importance score of each token can be compared between the source language and the target language according to a feature alignment result.

Interpreting cross-lingual models for natural language inference

A method provided for cross-lingual transfer trains a pre-trained multi-lingual language model based on a gold labeled training set in a source language to obtain a trained model. The method assigns each sample in an unlabeled target language set to a silver label according to a model prediction by the trained model to obtain set of silver labels, and performs uncertainty-aware label selection based on the silver label assigned to each sample according to the model prediction and the trained model to obtain selected silver labels. The method performs iterative training on the selected labels by applying the selected silver labels in the target language set as training labels and re-training the trained model with the gold labels and the selected silver labels to obtain an iterative model, and performs task-specific result prediction in target languages based on the iterative model to generate a final predicted result in target languages.

Self-learning framework of zero-shot cross-lingual transfer with uncertainty estimation

A computer-implemented method is provided for cross-lingual transfer. The method includes randomly masking a source corpus and a target corpus to obtain a masked source corpus and a masked target corpus. The method further includes tokenizing, by pretrained Natural Language Processing (NLP) models, the masked source corpus and the masked target corpus to obtain source tokens and target tokens. The method also includes transforming the source tokens and the target tokens into a source dependency parsing tree and a target dependency parsing tree. The method additionally includes inputting the source dependency parsing tree and the target dependency parsing tree into a graph encoder pretrained on a translation language modeling task to extract common language information for transfer. The method further includes fine-tuning the graph encoder and a down-stream network for a specific NLP down-stream task.

Cross-lingual zero-shot transfer via semantic and synthetic representation learning

Methods and systems for natural language processing include generating an encoder that includes a global part and a local part, where the global part encodes multi-hop relations between words in an input and where the local part encodes one-hop relations between words in the input. The encoder is trained to form a graph that represents tokens of an input text as nodes and that represents relations between the tokens as edges between the nodes.

Graph-based cross-lingual zero-shot transfer

Systems and methods are provided for automated computer code editing. The method includes training a code-editing neural network model using a corpus of code editing data samples, including the pre-editing samples and post-editing samples, and parsing the pre-editing samples and post-editing samples into an Abstract Syntax Tree (AST). The method further includes using a grammar specification to transform the AST tree into a unified Abstract Syntax Description Language (ASDL) graph for different programming languages, and using a gated graph neural network (GGNN) to compute a vector representation for each node in the unified Abstract Syntax Description Language (ASDL) graph. The method further includes selecting and aggregating support samples based on a query code with a multi-extent ensemble method, and altering the query code iteratively using the pattern learned from the pre- and post-editing samples.

Computer code refactoring

A method for employing a temporal context-aware question routing model (TCQR) in multiple granularities of temporal dynamics in community-based question answering (CQA) systems is presented. The method includes encoding answerers into temporal context-aware representations based on semantic and temporal information of questions, measuring answerers expertise in one or more of the questions as a coherence between the temporal context-aware representations of the answerers and encodings of the questions, modeling the temporal dynamics of answering behaviors of the answerers in different levels of time granularities by using multi-shift and multi-resolution extensions, and outputting answers of select answerers to a visualization device.

Temporal context-aware representation learning for question routing

Methods and systems of training and using a neural network model include training a time series embedding model and a text embedding model with unsupervised clustering to translate time series and text, respectively, to a shared latent space. The time series embedding model and the text embedding model are further trained using semi-supervised clustering that samples training data pairs of time series information and associated text for annotation.

Embedding multi-modal time series and text data

A computer-implemented method executed by at least one processor for software bug localization is presented. The method includes constructing a bug localization graph to capture relationships between bug tickets and relevant source code files from historical change-sets and an underlying source code repository, leveraging natural processing language tools to evaluate semantic similarity between a new bug ticket and a historical ticket, in response to the evaluated semantic similarity, for the new bug ticket, adding links between the new bug ticket a set of similar historical tickets, incorporating the new bug ticket in the bug localization graph, and developing a mathematical graph expression to determine a closeness relationship between the relevant source code files and the new bug ticket.

Growing community of inventors

Elkridge, MD, United States of America

Xuchao Zhang