Netskope, Inc.

Wayne Xin

Yi Zhang

Yihua Liao

Ari Azarafrooz

Niranjan Koduri

Emanoel Daryoush

Jason B Bryslawskyj

Image fingerprints (embeddings) are generated by an image fingerprinting model and indexed with an approximate nearest neighbors (ANN) model trained to identify the most similar fingerprint based on a subject embedding. For image matching, a score is provided that indicates a similarity between the input embedding and the most similar identified embedding, which allows for matching even when an image has been distorted, rotated, cropped, or otherwise modified. For image classification, the embeddings in the index are clustered and the clusters are labeled. Users can provide just a few images to add to the index as a labeled cluster. The ANN model returns a score and label of the most similar identified embedding for labeling the subject image if the score exceeds a threshold. As improvements are made to the image fingerprinting model, a converter model is trained to convert the original embeddings to be compatible with the new embeddings.

Image classification and accelerated classification training using deep learning image fingerprinting models and indexed embeddings

A system to generate an image classifier and test it nearly instantaneously is described herein. Image embeddings generated by an image fingerprinting model are indexed and an associated approximate nearest neighbors (ANN) model is generated. The embeddings in the index are clustered and the clusters are labeled. Users can provide just a few images to add to the index as a labeled cluster. The ANN model is trained to receive an image embedding as input and return a score and label of the most similar identified embedding. The label may be applied if the score exceeds a threshold value. The image classifier can be tested efficiently using Leave One Out Cross Validation ('LOOCV') to provide near-instantaneous quality indications of the image classifier to the user. Near-instantaneous indications of outliers in the provided images can also be provided to the user using a distance to the centroid calculation.

Image classifier automated testing and outlier detection

Image fingerprinting conversion between different image fingerprinting models

Image matching using deep learning image fingerprinting models and embeddings

Growing community of inventors

Santa Clara, CA, United States of America

Wayne Xin