Nec Corporation

Yuncong Chen

Haifeng Chen

Dongjin Song

Cristian Lumezanu

Wei Cheng

Zhengzhang Chen

Takehiko Mizoguchi

LuAn Tang

Yuji Kobayashi

Masanao Natsumeda

Wenchao Yu

Xuchao Zhang

Wei Zhu

Bo Dong

Takehiko Mizuguchi

Xiao Yu

Yanchi Liu

Shuchu Han

Xinyang Feng

Yuxiang Ren

Hao Yuan

Xiaoyuan Liang

Zexue He

Jurijs Nazarovs

Chuxu Zhang

A computer-implemented method for employing a time-series-to-text generation model to generate accurate description texts is provided. The method includes passing time series data through a time series encoder and a multilayer perceptron (MLP) classifier to obtain predicted concept labels, converting the predicted concept labels, by a serializer, to a text token sequence by concatenating an aspect term and an option term of every aspect, inputting the text token sequence into a pretrained language model including a bidirectional encoder and an autoregressive decoder, and using adapter layers to fine-tune the pretrained language model to generate description texts.

Concept-conditioned and pretrained language models based on time series to free-form text description generation

Methods and systems for anomaly detection include determining whether a system is in a stable state or a dynamic state based on input data from one or more sensors in the system, using reconstruction errors from a respective stable model and dynamic model. It is determined that the input data represents anomalous operation of the system, responsive to a determination that the system is in a stable state, using the reconstruction errors. A corrective operation is performed on the system responsive to a determination that the input data represents anomalous operation of the system.

Anomaly detection on dynamic sensor data

Methods and systems for training a neural network include collecting model exemplar information from edge devices, each model exemplar having been trained using information local to the respective edge devices. The collected model exemplar information is aggregated together using federated averaging. Global model exemplars are trained using federated constrained clustering. The trained global exemplars are transmitted to respective edge devices.

Federated learning for anomaly detection

Edge-side federated learning for anomaly detection

Systems and methods for data fusion and analysis of vehicle sensor data, including receiving a multiple modality input data stream from a plurality of different types of vehicle sensors, determining latent features by extracting modality-specific features from the input data stream, and aligning a distribution of the latent features of different modalities by feature-level data fusion. Classification probabilities can be determined for the latent features using a fused modality scene classifier. A tree-organized neural network can be trained to determine path probabilities and issue driving pattern judgments, with the tree-organized neural network including a soft tree model and a hard decision leaf. One or more driving pattern judgments can be issued based on a probability of possible driving patterns derived from the modality-specific features.

Data fusion and analysis engine for vehicle sensors

A method classifies missing labels. The method computes, using a neural network model trained on training data, rank-based statistics of a feature of a time series segment to attempt to select two candidate labels from the training data that the segment most likely belongs to. The method classifies the segment using k-NN-based classification applied to the training data, responsive to the two candidate labels being present in the training data. The method classifies the segment by hypothesis testing, responsive to only one candidate label being present in the training data. The method classifies the segment into a class with higher values of the rank-based statistics from among a plurality of classes with different values of the rank-based statistics, responsive to no candidate labels being present in the training data. The method corrects a prediction by an applicable one of the classifying steps by majority voting with time windows.

Ordinal time series classification with missing information

A computer-implemented method for multi-model representation learning is provided. The method includes encoding, by a trained time series (TS) encoder, an input TS segment into a TS-shared latent representation and a TS-private latent representation. The method further includes generating, by a trained text generator, a natural language text that explains the input TS segment, responsive to the TS-shared latent representation, the TS-private latent representation, and a text-private latent representation.

Co-disentagled series/text multi-modal representation learning for controllable generation

A method for system metric prediction and influential events identification by concurrently employing metric logs and event logs is presented. The method includes concurrently modeling multivariate metric series and individual events in event series by a multi-stream recurrent neural network (RNN) to improve prediction of future metrics, where the multi-stream RNN includes a series of RNNs, one RNN for each metric and one RNN for each event sequence and modeling causality relations between the multivariate metric series and the individual events in the event series by employing an attention mechanism to identify target events most responsible for fluctuations of one or more target metrics.

Causal attention-based multi-stream RNN for computer system metric prediction and influential events identification based on metric and event logs

Methods and systems of training and using a neural network model include training a time series embedding model and a text embedding model with unsupervised clustering to translate time series and text, respectively, to a shared latent space. The time series embedding model and the text embedding model are further trained using semi-supervised clustering that samples training data pairs of time series information and associated text for annotation.

Embedding multi-modal time series and text data

A method for multivariate time series prediction is provided. Each time series from among a batch of multiple driving time series and a target time series is decomposed into a raw component, a shape component, and a trend component. For each decomposed component, select a driving time series relevant thereto from the batch and obtain hidden features of the selected driving time series, by applying the batch to an input attention-based encoder of an Ensemble of Clustered dual-stage attention-based Recurrent Neural Networks (EC-DARNNS). Automatically cluster the hidden features in a hidden space using a temporal attention-based decoder of the EC-DARNNS. Each Clustered dual-stage attention-based RNN in the Ensemble is dedicated and applied to a respective one of the decomposed components. Predict a respective value of one or more future time steps for the target series based on respective prediction outputs for each of the decomposed components by the EC-DARNNS.

Ensemble of clustered dual-stage attention-based recurrent neural networks for multivariate time series prediction

Systems and methods for updating a classification model of a neural network. The methods include selecting, as a set of landmarks, a limited number of data from a set of historical data used to train a classification model. Additionally, the methods generate new training data from recently collected data. Further, the methods update the classification model with the new training data and the set of landmarks to obtain an updated classification model having a loss function configured to capture similarities in the new training data and remember similarities in the historical data represented by the set of landmarks within a predefined tolerance.

Landmark-based classification model updating

Methods and systems for predicting failure in a cyber-physical system include determining a prediction index based on a comparison of input time series, from respective sensors in a cyber-physical system, to failure precursors. A failure precursor is detected in the input time series, responsive to a comparison of the prediction index to a threshold. A subset of the sensors associated with the failure precursor is determined, based on a gradient of the prediction index. A corrective action is performed responsive to the determined subset of sensors.

Failure prediction using gradient-based sensor identification

Systems and methods for detecting anomaly in video data are provided. The system includes a generator that receives past video frames and extracts spatio-temporal features of the past video frames and generates frames. The generator includes fully convolutional transformer based generative adversarial networks (FCT-GANs). The system includes an image discriminator that discriminates generated frames and real frames. The system also includes a video discriminator that discriminates generated video and real video. The generator trains a fully convolutional transformer network (FCTN) model and determines an anomaly score of at least one test video based on a prediction residual map from the FCTN model.

Fully convolutional transformer based generative adversarial networks

A system for cross-modal data retrieval is provided that includes a neural network having a time series encoder and text encoder which are jointly trained using an unsupervised training method which is based on a loss function. The loss function jointly evaluates a similarity of feature vectors of training sets of two different modalities of time series and free-form text comments and a compatibility of the time series and the free-form text comments with a word-overlap-based spectral clustering method configured to compute pseudo labels for the unsupervised training method. The computer processing system further includes a database for storing the training sets with feature vectors extracted from encodings of the training sets. The encodings are obtained by encoding a training set of the time series using the time series encoder and encoding a training set of the free-form text comments using the text encoder.

Word-overlap-based clustering cross-modal retrieval

Systems and methods for anomaly detection are provided. The method includes structuring a multi-channel spatial-temporal sequence as a four-dimensional array. The method also includes decomposing the four-dimensional array to form a low-rank component representing a background signal and a residual component representing anomalies for each time point of the multi-channel spatial-temporal sequence. The method further includes determining a sequence of anomaly maps by stacking the residual components at all time points together. Anomalies are identified based on the sequence of anomaly maps.

Growing community of inventors

Plainsboro, NJ, United States of America

Yuncong Chen