International Business Machines Corporation

David Darden Chambliss

Joseph S Glider

Leah Shalev

Ehood Garmiza

Prashant Pandey

Harry M Yudenfriend

Maohua Lu

Mihail Corneliu Constantinescu

Divyesh Jadav

James Alan Ruddy

Danny Harnik

Jian J Xu

Tzongyu Paul Lee

Ramachandran Gopalakrishna Menon

James Lee Hafner

Rui Zhang

John Glenn Thompson

Ety Khaitzin

Joshua W Knight

Eliyahu Weissbrem

Ronald K Kreuzenstein

John J Lee

Matthew J Kalos

Itzhack Goldberg

Neil Sondhi

Lawrence Yiumchee Chiu

Yosef Shatsky

Frank William Brice

Tarun Thakur

Bhushan Pradip Jain

Guillermo Alvarez

Jeou-Rong Lay

Luis Alfonso Lastras-Montano

Bulent Abali

Jonathan Amit

Nimrod Megiddo

Yu-Cheng Hsu

Chaim Koifman

Bruce McNutt

Wendy Ann Belluomini

Aviv Caro

Chiahong Chen

William Griswold Sherman

Himabindu Pucha

Cameron James Mcallister

Ronald Franklin Marks

Eyal Lotem

Robin Frederick Farrow

John G Aschoff

Eric Joseph Wood

Eric Dean Tober

Dilip N Simha

Peter Clayton Belknap

Leonard Chi Chiu Chung

Rhonda L Childress

Vijayshankar Raman

Howard Newton Martin

Haim Helman

Kaladhar Voruganti

Garret Frederick Swart

Jaishankar Moothedath Menon

Joseph S Hyde, Ii

Dmitry Sotnikov

Oded Margalit

Aayush Gupta

Glen Hattrup

Lei Liu

Jonathan Fischer-Toubol

Mark Kunitomi

Nicholas C Matsakis

Michael Moser

Bruce Light Hillsberg

Yifat Kuttner

Hyojun Kim

Ohad Rodeh

Nir Mor

Cliff Pickover

Forest Dussault

Information received from a graphical user interface (GUI) and a list of user-curated command line patterns are received by an auto-wrapper system, wherein the auto-wrapper system is associated with an analytics workflow service. A module including a parameter space having one or more parameters and options used in the list of user-curated command line patterns is generated, by the auto-wrapper system, wherein content for each parameter is derived from the parameter's presence in the list of user-curated command line patterns combined with the information received from a GUI.

Auto-wrappering tools with guidance from exemplar commands

Computer program products, as well as corresponding systems and methods are configured for performing deduplication in conjunction with random read and write operations, and include: computing a fingerprint of data included in a write request; determining whether a short term dictionary comprises an entry corresponding to the fingerprint; in response to determining the short term dictionary comprises the entry corresponding to the fingerprint, writing the data to a data store in a deduplicating manner; in response to determining the short term dictionary does not comprise the entry, determining whether a long term dictionary corresponding to the namespace comprises the entry; in response to determining the long term dictionary comprises the entry, writing the data to the data store in the deduplicating manner; and in response to determining the long term dictionary does not comprise the entry, writing the data to the data store in a non-deduplicating manner.

Region-integrated data deduplication implementing a multi-lifetime duplicate finder

In one embodiment, a deduplicating storage system includes a processor and logic integrated with and/or executable by the processor. The logic is configured to cause the processor to perform a method which includes: computing a fingerprint of a data chunk, and determining whether a short term dictionary corresponding to the namespace comprises an entry corresponding to the fingerprint. In response to determining the short term dictionary does not comprise the entry, a determination is made whether a long term dictionary corresponding to the namespace comprises the entry. In response to determining the long term dictionary comprises the entry: the data chunk is written to the data store in the deduplicating manner, and the short term dictionary is repopulated with the entry. Moreover, in response to determining the long term dictionary does not comprise the entry, the data chunk is written to the data store in a non-deduplicating manner.

Embodiments of the invention relate to a storage system organized into a hierarchy of storage tiers, with at least one tier reflecting a high performance tier and at least one tier reflecting a lower performance tier. The high performance tier has a capacity restriction and has a limited quantity of blocks and pages may be placed in the tier. Assessments are conducted and a preferred selection of blocks and pages are recommended for placement; the recommendation is based on the assessment. The recommendation is converted to an actual placement, resulting in placement of at least one block, an in one embodiment at least one page, in the high performance tier.

Optimizing performance of tiered storage

Computer program products, as well as corresponding systems and methods are configured for performing deduplication in conjunction with random read and write operations, and include: receiving a write request comprising data; computing a fingerprint of the data; determining whether a short term dictionary comprises an entry corresponding to the fingerprint; in response to determining the short term dictionary comprises the entry corresponding to the fingerprint, writing the data to a data store in a deduplicating manner; in response to determining the short term dictionary does not comprise the entry, determining whether a long term dictionary corresponding to the namespace comprises the entry; in response to determining the long term dictionary comprises the entry, writing the data to the data store in the deduplicating manner; and in response to determining the long term dictionary does not comprise the entry, writing the data to the data store in a non-deduplicating manner.

In one embodiment, a computer-implemented method for performing deduplication in conjunction with random read and write operations across a namespace that is divided into a plurality of disjoint regions is disclosed. The method includes: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing a deduplicated write operation. The deduplicated write operation includes: determining whether the disjoint regions comprise a second data chunk identical in content to a first data chunk; and in response to determining one or more of the plurality of disjoint regions other than the first region includes the second data chunk, establishing a reference from the first data chunk to the second data chunk. Corresponding systems and computer program products are also disclosed.

Region-integrated data deduplication

Embodiments relate to host-side cache migration. An aspect is a method that includes determining pre-fetch hints associated with a source cache that is local to a source host machine. The source cache includes pages of cache data for a virtual machine. The pre-fetch hints are sent to a pre-fetch planner to create a pre-fetch plan. The pre-fetch hints are sent based on migration of the virtual machine to a target host machine including a target cache that is local. At the source host machine, a cache migration request is received based on the pre-fetch plan. A first subset of the pages is sent from the source cache through a host-to-host communication channel to the target cache based on the cache migration request. A second subset of the pages is sent from the source cache through a host-storage communication channel to a shared storage to be relayed to the target cache.

Host-side cache migration

Embodiments of the invention relate to storage systems, and for recording event records into a log. Different statistics may be tallied from the log, with the different statistics generating different statistical measurements. The aspect of processing of I/O and preparation of statistical measurements are separated, thereby enabling data structures and algorithms which would not be considered feasible for use as part of I/O processing, because they may compromise the performance or other essential characteristic of I/O processing, to be employed.

Storage system with trace-based management

A method, including partitioning a dataset into a first number of data units, and selecting, based on a sampling ratio, a second number of the data units. A hash value is calculated for each of the selected data units, and a first histogram is computed indicating a first duplication count for each of the calculated hash values. Based on respective frequencies of the calculated hash values, a second histogram is computed indicating an observed frequency for each of the first duplication counts in the first histogram, and based on the sampling ratio and the second histogram, a target function is derived. A third histogram that minimizes the target function is derived, the third histogram including, for the first number of the storage units, second duplication counts and a respective predicted frequency for each of the second duplication counts. Finally, a deduplication ratio is determined based on the third histogram.

Sampling-based deduplication estimation

A computer-implemented method for determining performance shortfall in a storage system includes recording a log of I/O operations of a storage system, wherein each I/O operation includes associated operational parameters. The method prepares the storage system to perform a replay of a performance shortfall that occurred during a specified time frame. The method performs the replay on the storage system based at least on the log of I/O operations and the I/O operations' associated operational parameters for the specified time frame. The method identifies a cause for the performance shortfall during the specified time frame based at least on the difference between I/O performance measures of the replay and I/O performance measures for the specified time frame.

Identification of storage performance shortfalls

A method and system for regulating access by an access program to a data object residing in a storage system, which may be used to protect against data theft in a storage server. The storage server receives, from a client node, a certificate request for a certificate pertaining to access of the data object by the access program. The storage server validates the certificate request and in response, generates the certificate and transmits the certificate to the client node. The certificate request and the certificate each include a signature of the access program and an identifier of the data object. The storage server receives from the client node an I/O request for access of the data object by the access program. The storage server determines whether the I/O request is valid or invalid and processes the I/O request with privileged handling or degraded handling, respectively.

Data theft deterrence

Embodiments relate to avoiding out-of-space conditions in storage controllers operating with efficiency capabilities between virtual space in a data container and real space in a storage container. Both the real space and the virtual space are monitored. The real space usage is compared to a threshold to provide information about occupancy of the real space. A virtual size of the virtual object is adjusted responsive to the real space usage meeting or exceeding the threshold.

Avoiding out-of-space conditions in storage controllers

In one embodiment, a computer program product is configured for performing deduplication in conjunction with random read and write operations across a namespace divided into a plurality of disjoint regions. The computer program product includes a computer readable storage medium having program instructions embodied therewith, where the computer readable storage medium is not a transitory signal per se. The program instructions are executable by a computer to cause the computer to perform a method including: maintaining a metadata structure for each of the plurality of disjoint regions via the respective region manager(s) of the plurality of disjoint regions, each metadata structure comprising metadata indicating a physical storage location of one or more data chunks associated with the respective region; and performing, by the computer, a deduplicated write operation of a first data chunk in a first region of the plurality of disjoint regions.

Embodiments relate to sub-block input/output (I/O) commands in a computer storage device. An aspect includes receiving a plurality of I/O commands on an I/O interface of the computer storage device, the computer storage device comprising a byte stream buffer module comprising a plurality of byte stream buffers, a block buffer module comprising a plurality of block buffers, and a main storage comprising a plurality of fixed-size storage blocks. Another aspect includes processing the plurality of I/O commands by the computer storage device. Another aspect includes a block write command comprising a command to write an amount of write data having a size equal to one of the plurality of a fixed-size storage block. Another aspect includes a sub-block write command comprising a command to write an amount of write data having a size that is less than a size of a fixed-size storage block.

Sub-block input/output (I/O) commands for storage device including byte stream buffer

Growing community of inventors

Morgan Hill, CA, United States of America

David Darden Chambliss