High dimensional dense tensor representation for log data

Abstract

In some implementations, a device may obtain a training corpus, from a set of pre-processed log data, associated with an alphanumeric format. The device may encode the training corpus to obtain encoded data using a set of tokens. The device may calculate a sequence length based on a statistical parameter associated with the training corpus. The device may generate a set of input sequences and a set of target sequences based on the encoded data, where each input sequence and each target sequence has a length equal to the sequence length. The device may generate a training data set based on combining the set of input sequences and the set of target sequences. The device may train a deep neural network (DNN) using the training data set and based on one or more hyperparameters to obtain a set of embedding tensors associated with an embedding layer of the DNN.