Systems and methods for de-identifying and transforming data representations for data queries while maintaining data privacy

Abstract

De-identification and transformation systems and methods are provided. Raw data may be tokenized to raw tokens may include a controlled infinite vocabulary of standardized tokens with materialized uncertainty. The raw tokens may be converted to raw hash values of an infinite vocabulary, which may be joined across the multiple data sources to create synthetic composite hashed records ('SynCHRs'). A raw embedding may be generated using a woven encrypted locality-sensitive hashing (“WELSH”) process. The raw embedding may comprise an array of floating point numbers representing the SynCHRs in a vector space that may not be mapped in reverse. One or more SynCHRs having a similarity to a target record may be identified via an indexed vector search. Features of the one or more SynCHRs may be aggregated and appended to the target record, without identifying the record or records used to create the aggregated feature internally or externally to the process.