International Business Machines Corporation

Amit Schreiber

Lior Aronovich

Asaf Levy

Samuel Krikler

Amir Kredi

An original deduplication file system (DFS) file is partitioned into a first set of sections being sections including data affected by rewrite operations and a second set of sections being sections including data unaffected by rewrite operations. A new DFS file to be stored as part of a plurality of user files is created, the plurality of user files including the original DFS file and being accessible by a sequential DFS and a dynamic non-DFS, the dynamic non-DFS including a plurality of dynamic metadata files having entries pointing to corresponding sections of the original DFS files. The first set of sections of the original DFS file including data affected by rewrite operations is directly written into the new DFS file. The second set of sections from the original DFS file including data unaffected by rewrite operations is quoted into the new DFS file. The original DFS file is deleted.

Dynamic rewrite of files within deduplication system

A dynamic layer above a sequential deduplication file system (denoted as DFS) implements the rewrite functionality. A user file is composed of one or more DFS files. As incoming data is written into a user file, the data is written by the dynamic layer sequentially into DFS files, created one by one. For each user file this dynamic layer creates and maintains a dynamic metadata file, in a regular, non deduplicated file system. This metadata file contains entries pointing to sections of DFS files.

Various embodiments writing data are provided. In one embodiment, the data arranged in a plurality of write intervals is loaded into a plurality of buffers, the totality of the plurality of buffers configured as a sliding write window mechanism adapted for movement to accommodate the write intervals. The data may reach the storage system out of a sequential order, and by loading it appropriately into the said buffers the data is ordered sequentially before it is written to the storage media. When a commencing section of the sliding write window is filled up with written data, this section is flushed to the storage media, and the window slides forward, to accommodate further data written by the writers. The writers are synchronized with the interval reflected by the current position of the sliding write window, and they send data to be written only where this data fits into the current interval of the window.

Growing community of inventors

Kiriat Ata, Israel

Amit Schreiber