Netskope, Inc.

Ari Azarafrooz

Yihua Liao

Yi Zhang

Zhi Xu

Niranjan Koduri

Emanoel Daryoush

Jason B Bryslawskyj

Najmeh Miramirkhani

Wayne Xin

Siying Yang

Raymond Joseph Canzanese, Jr

Zhenxin Zhan

Xinjun Zhang

Ghanashyam Satpathy

Dagmawi Mulugeta

Hung-Ming Chen

Image fingerprints (embeddings) are generated by an image fingerprinting model and indexed with an approximate nearest neighbors (ANN) model trained to identify the most similar fingerprint based on a subject embedding. For image matching, a score is provided that indicates a similarity between the input embedding and the most similar identified embedding, which allows for matching even when an image has been distorted, rotated, cropped, or otherwise modified. For image classification, the embeddings in the index are clustered and the clusters are labeled. Users can provide just a few images to add to the index as a labeled cluster. The ANN model returns a score and label of the most similar identified embedding for labeling the subject image if the score exceeds a threshold. As improvements are made to the image fingerprinting model, a converter model is trained to convert the original embeddings to be compatible with the new embeddings.

Image classification and accelerated classification training using deep learning image fingerprinting models and indexed embeddings

A system to generate an image classifier and test it nearly instantaneously is described herein. Image embeddings generated by an image fingerprinting model are indexed and an associated approximate nearest neighbors (ANN) model is generated. The embeddings in the index are clustered and the clusters are labeled. Users can provide just a few images to add to the index as a labeled cluster. The ANN model is trained to receive an image embedding as input and return a score and label of the most similar identified embedding. The label may be applied if the score exceeds a threshold value. The image classifier can be tested efficiently using Leave One Out Cross Validation ('LOOCV') to provide near-instantaneous quality indications of the image classifier to the user. Near-instantaneous indications of outliers in the provided images can also be provided to the user using a distance to the centroid calculation.

Image classifier automated testing and outlier detection

Image fingerprinting conversion between different image fingerprinting models

A cloud-based network security system (NSS) is described. The NSS uses a sandbox to safely detonate and extract information about a document and uses machine learning algorithms to analyze the information to predict whether the document contains malicious software. Specifically, during the detonation, static and dynamic information about the document is captured in the sandbox as well as character strings from images in the document. The dynamic information (and sometimes the static information) is input to an AI or machine learning model trained to provide an output indicating a prediction of whether the document contains malware. The character strings are compared with a batch of phishing keywords to generate a heuristic score. A validation engine combines the output from the AI or machine learning model and the heuristic score to classify the document as malicious or clean. Security policies can then be applied based on the classification.

Machine learning powered cloud sandbox for malware detection

Disclosed is phishing classifier that classifies a URL and content page accessed via the URL as phishing or not is disclosed, with URL feature hasher that parses and hashes the URL to produce feature hashes, and headless browser to access and internally render a content page at the URL, extract HTML tokens, and capture an image of the rendering. Also disclosed are an HTML encoder, trained on HTML tokens extracted from pages at URLs, encoded, then decoded to reproduce images captured from rendering, that produces an HTML encoding of the tokens extracted, and an image embedder, pretrained on images, that produces an image embedding of the image captured. Further, phishing classifier layers, trained on the feature hashes, the HTML encoding, and the image embedding, process the URL feature hashes, HTML encoding and image embeddings to produce a likelihood score that the URL and the page accessed presents a phishing risk.

Detecting phishing websites via a machine learning-based system using URL feature hashes, HTML encodings and embedded images of content pages

Image matching using deep learning image fingerprinting models and embeddings

The disclosed technology teaches facilitate User and Entity Behavior Analytics (UEBA) by classifying a file being transferred as encrypted or not. The technology involves monitoring movement of a files by a user over a wide area network, detecting file encryption for the files using a trained classifier, wherein the detecting includes processing by the classifier some or all of the following features extracted from each of the files: a chi-square randomness test; an arithmetic mean test; a serial correlation coefficient test; a Monte Carlo-Pi test; and a Shannon entropy test, counting a number of the encrypted files moved by the user in a predetermined period, comparing a predetermined maximum number of encrypted files allowed in the predetermined period to the count of the encrypted files moved by the user and detecting that the user has moved more encrypted files than the predetermined maximum number, and generating an alert.

ML-based encrypted file classification for identifying encrypted data movement

Disclosed is classifying a URL and a page accessed via the URL as phishing or not. URL embedder extracts characters in a predetermined set from the URL to produce a character string trained using ground truth classification of the URL, producing a URL embedding. HTML parser accesses content at the URL and extracts HTML tokens from the page. Further, HTML encoder, trained on HTML tokens extracted from pages at example URLs, each example URL accompanied by a ground truth image captured from the page accessed via the example URL, produces an HTML encoding of the extracted tokens. Also, phishing classifier layers, trained on the URL embedding and the HTML encoding of example URLs, processes a concatenated input of the URL embedding and the HTML encoding to produce a score of a phishing risk.

Machine learning-based system for detecting phishing websites using the URLS, word encodings and images of content pages

Disclosed is classifying a URL and a content page accessed via the URL as phishing or not. URL embedder extracts characters in a predetermined set from the URL to produce a character string trained using ground truth classification of the URL, producing a URL embedding. HTML parser accesses content at the URL and extracts HTML tokens from the content page. Further, HTML encoder, trained on HTML tokens extracted from content pages at example URLs, each example URL accompanied by a ground truth image captured from the content page accessed via the example URL, produces an HTML encoding of the tokens extracted from the page. Also, phishing classifier layers, trained on the URL embedding and the HTML encoding of example URLs, processes a concatenated input of the URL embedding and the HTML encoding to produce a likelihood score that the URL and content accessed via the URL presents a phishing risk.

Growing community of inventors

Rancho Santa Margarita, CA, United States of America

Ari Azarafrooz