Nonvolatile memory systems configured to use deduplication and methods of controlling the same

Abstract

Systems and methods of determining a similarity between data units in a nonvolatile memory are disclosed. One method includes obtaining first and second data units and dividing the first and second data units into a first plurality of non-overlapping chunks of data and a second plurality of non-overlapping chunks of data. The method further includes determining a first plurality of values and a second plurality of values associated with the chunks, and determining a similarity between the first second data units based on the first plurality values and of the second plurality of values. In one example embodiment, a similarity between an incoming data unit and another data unit is determined based on the number of buckets storing an incoming index value and another index value associated with the another data unit. A plurality of buckets in a table is determined based on a selected plurality of hash values.