Transformation of embedding matrices for compression, storage, and reuse

Abstract

The present disclosure relates to systems, non-transitory computer-readable media, and methods for preparing embedding matrices for compression, storage, and reuse. For example, the disclosed systems can receive or generate an embedding matrix comprising a plurality of embedding vectors representing a text content item, the plurality of embedding vectors exhibiting locality in a first dimension of the embedding matrix. The disclosed systems can apply a decorrelating transform along the first dimension of the embedding matrix to generate a transformed embedding matrix by compacting energy distributed across the embedding matrix in the first dimension into a sub-portion of the transformed embedding matrix. Further, the disclosed systems can compress the transformed embedding matrix by performing a compression operation to generate a compressed embedding matrix, and can store the compressed embedding matrix for the text content item.