Shared data encryption and confidentiality

Abstract

Embodiments of the invention relate to processing streams of encrypted data received from multiple users. As the streams are processed, smaller partitions in the form of data chunks are created and subject to individual decryption. The data chunks are placed into sub-stream based on a master key associated with its owning entity. Prior to processing, the data chunks in each stream are decrypted, and advanced functions, including but not limited to de-duplication and compression, are individually applied to the data chunks, followed by aggregation of processed data chunks into data units and encryption of the individual data units including use of a master key from the data's owning entity. Individual encryption units are created by encrypting the data unit(s) with an encryption key, thereby limiting access to the data unit. Confidentiality of data is maintained, and the ability of storage systems to perform data reduction functions is supported.