Oracle International Corporation

Sanjay Jinturkar

Nipun Agarwal

Karoon Rashedi Nia

Tayler Hetherington

Zahra Zohrevand

Yasha Pushak

Hamed Ahmadi

Venkatanathan Varadarajan

Jian Wen

Shrikumar Hariharasubrahmanian

Ali Moharrer

Sandeep Agrawal

Hesam Fathi Moghadam

Anatoly Yakovlev

Jingxiao Cai

Nikan Chavoshi

Vaseem Akram

Nishesh Rai

Reema Hingorani

Lijian Wan

Herein are techniques for topic modeling and content perturbation that provide machine learning (ML) explainability (MLX) for natural language processing (NLP). A computer hosts an ML model that infers an original inference for each of many text documents that contain many distinct terms. To each text document (TD) is assigned, based on terms in the TD, a topic that contains a subset of the distinct terms. In a perturbed copy of each TD, a perturbed subset of the distinct terms is replaced. For the perturbed copy of each TD, the ML model infers a perturbed inference. For TDs of a topic, the computer detects that a difference between original inferences of the TDs of the topic and perturbed inferences of the TDs of the topic exceeds a threshold. Based on terms in the TDs of the topic, the topic is replaced with multiple, finer-grained new topics. After sufficient topic modeling, a regional explanation of the ML model is generated.

Efficient and accurate regional explanation technique for NLP models

End-to-end explanation techniques, which efficiently explain the behavior (feature importance) of any machine learning model on large tabular datasets, are disclosed. These techniques comprise two down-sampling methods to efficiently select a small set of representative samples of a high-dimensional dataset for explaining a machine learning model by making use of the characteristics of the dataset or of an explainer of a machine learning model to optimize the explanation quality. These techniques significantly improve the explanation speed while maintaining the explanation quality of a full dataset evaluation.

Efficient and scalable computation of global feature importance explanations

A systematic explainer is described herein, which comprises local, model-agnostic, surrogate ML model-based explanation techniques that faithfully explain predictions from any machine learning classifier or regressor. The systematic explainer systematically generates local data samples around a given target data sample, which improves on exhaustive or random data sample generation algorithms. Specifically, using principles of locality and approximation of local decision boundaries, techniques described herein identify a hypersphere (or data sample neighborhood) over which to train the surrogate ML model such that the surrogate ML model produces valuable, high-quality information explaining data samples in the neighborhood of the target data sample. Combining this systematic local data sample generation and a supervised neighborhood selection approach to weighting generated data samples relative to the target data sample achieves high explanation fidelity, locality, and repeatability when generating explanations for specific predictions from a given model.

Systematic approach for explaining machine learning predictions

Herein, a computer generates and evaluates many preprocessor configurations for a window preprocessor that transforms a training timeseries dataset for an ML model. With each preprocessor configuration, the window preprocessor is configured. The window preprocessor then converts the training timeseries dataset into a configuration-specific point-based dataset that is based on the preprocessor configuration. The ML model is trained based on the configuration-specific point-based dataset to calculate a score for the preprocessor configuration. Based on the scores of the many preprocessor configurations, an optimal preprocessor configuration is selected for finally configuring the window preprocessor, after which, the window preprocessor can optimally transform a new timeseries dataset such as in an offline or online production environment such as for real-time processing of a live streaming timeseries.

Automated machine learning pipeline for timeseries datasets utilizing point-based algorithms

A model-agnostic global explainer for textual data processing (NLP) machine learning (ML) models, 'NLP-MLX', is described herein. NLP-MLX explains global behavior of arbitrary NLP ML models by identifying globally-important tokens within a textual dataset containing text data. NLP-MLX accommodates any arbitrary combination of training dataset pre-processing operations used by the NLP ML model. NLP-MLX includes four main stages. A Text Analysis stage converts text in documents of a target dataset into tokens. A Token Extraction stage uses pre-processing techniques to efficiently pre-filter the complete list of tokens into a smaller set of candidate important tokens. A Perturbation Generation stage perturbs tokens within documents of the dataset to help evaluate the effect of different tokens, and combinations of tokens, on the model's predictions. Finally, a Token Evaluation stage uses the ML model and perturbed documents to evaluate the impact of each candidate token relative to predictions for the original documents.

Global, model-agnostic machine learning explanation technique for textual data

Techniques are described for fast approximate conditional sampling by randomly sampling a dataset and then performing a nearest neighbor search on the pre-sampled dataset to reduce the data over which the nearest neighbor search must be performed and, according to an embodiment, to effectively reduce the number of nearest neighbors that are to be found within the random sample. Furthermore, KD-Tree-based stratified sampling is used to generate a representative sample of a dataset. KD-Tree-based stratified sampling may be used to identify the random sample for fast approximate conditional sampling, which reduces variance in the resulting data sample. As such, using KD-Tree-based stratified sampling to generate the random sample for fast approximate conditional sampling ensures that any nearest neighbor selected, for a target data instance, from the random sample is likely to be among the nearest neighbors of the target data instance within the unsampled dataset.

Fast, approximate conditional distribution sampling

Herein is a probabilistic indexing technique for searching semi-structured text documents in columnar storage formats such as Parquet, using columnar input/output (I/O) avoidance, and needing minimal storage overhead. In an embodiment, a computer associates columns with text strings that occur in semi-structured documents. Text words that occur in the text strings are detected. Respectively for each text word, a bitmap, of a plurality of bitmaps, that contains a respective bit for each column is generated. Based on at least one of the bitmaps, some of the columns or some of the semi-structured documents are accessed.

Probabilistic text index for semi-structured data in columnar analytics storage formats

Herein are machine learning (ML) techniques for unsupervised training with a corpus of signaling system 7 (SS7) messages having a diversity of called and calling parties, operation codes (opcodes) and transaction types, numbering plans and nature of address indicators, and mobile country codes and network codes. In an embodiment, a computer stores SS7 messages that are not labeled as anomalous or non-anomalous. Each SS7 message contains an opcode and other fields. For each SS7 message, the opcode of the SS7 message is stored into a respective feature vector (FV) of many FVs that are based on respective unlabeled SS7 messages. The FVs contain many distinct opcodes. Based on the FVs that contain many distinct opcodes and that are based on respective unlabeled SS7 messages, an ML model such as a reconstructive model such as an autoencoder is unsupervised trained to detect an anomalous SS7 message.

Anomaly detection in SS7 control network using reconstructive neural networks

Techniques are described herein for indexing personal information in columnar data storage format based files. In an embodiment, row groups of rows that comprise a plurality of columns are stored in a set of files. Each column of a row group is stored in a chunk of column pages in the set of files. A regular expression index that indexes a particular column in the set of files is stored for each row group. The regular expression index identifies column pages in the chunk of the particular column that include a particular column value that satisfies a regular expression specified in a query. The regular expression specified in the query in evaluated against the particular column using the regular expression index.

Growing community of inventors

Santa Clara, CA, United States of America

Sanjay Jinturkar