Emc Corporation

Emc IP Holding Company LLC

Other

Kenneth J Taylor

Adrian Michaud

Roy E Clark

Randall Shain

Junping Zhao

Stephen Wing-Kin Au

Yaron Dar

Kun Wang

Bradford B Glade

Balakrishnan Ganeshan

Kevin Xu

Gregory S Robidoux

Txomin Barturen

Robert P Ng

Kenneth Durazzo

John S Harwood

Amnon Naamad

Barry A Burke

Sen Zhang

Lin Peng

Lance Fang

Rollin E Taylor

Wing-Kin Au

In a distributed data processing system with a set of multiple nodes, a first data shuffle memory pool is maintained at a data shuffle writer node, and a second data shuffle memory pool is maintained at a data shuffle reader node. The data shuffle writer node and the data shuffle reader node are part of the set of multiple nodes of the distributed data processing system. In-memory compression is performed on at least a portion of a data set from the first data shuffle memory pool. At least a portion of the compressed data is transmitted from the first data shuffle memory pool to the second data shuffle memory pool in a peer-to-peer manner. Each of the first data shuffle memory pool and the second data shuffle memory pool may include a hybrid memory configuration.

Memory optimized data shuffle

In a method of tiering data in a computing environment, data service status information is obtained at a first storage environment regarding a data service operation performed on one or more data sets that qualify for storage at a second storage environment based on a given policy, and, based on the obtained data service status information, a determination is made to store the one or more data sets at the second storage environment without performing a reverse operation of the data service operation on the one or more data sets.

Compression and encryption aware optimized data moving across a network

Data service status information is obtained at a local storage tier regarding a data service operation performed on one or more data sets that qualify for remote storage based on a given policy. A determination is made whether or not to store at least a portion of the one or more data sets at a remote storage tier based on the obtained data service status information.

Data tiering based on data service status

Methods and apparatus for providing for a cache replacement policy for page caches for storage having a first memory tier having regions and virtual memory having mmaps of ones of the regions in the first memory tier. In an embodiment, the cache replacement policy includes setting a color hint to a first one of the cached pages, wherein the color hint includes a value indicating hotness of the first one of the cached pages.

Methods and apparatus for memory tier page cache coloring hints

A system, computer program product, and computer-executable method of managing one or more tiers of memory of a host computing system, the system, computer program product, and computer-executable method including accessing a portion of raw data from a memory page associated with data stored on the one or more tiers of memory, sampling the portion of raw data to select a sample data, analyzing the sample data to determine a sample category, and classifying the portion of raw data based at least in part by considering the sample category.

Classification and identification from raw data within a memory domain

Transparent, fine-grained, and adaptive data compression is described. A system determines a first data chunk and a second data chunk in a persistent storage. The system determines a first data read count and/or a first data write count for the first data chunk, and a second data read count and/or a second data write count for the second data chunk. The system determines then a first data compression status for the first data chunk and a second data compression status for the second data chunk. Based on the first data compression status and second data compression status, the system stores data in the first data chunk and data in the second data chunk to the persistent storage.

Transparent, fine-grained and adaptive data compression

Systems, methods, and articles of manufacture comprising processor-readable storage media are provided to implement read-ahead memory operations using learned memory access patterns for memory management systems. For example, a method for managing memory includes receiving a request from requestor (e.g., an active process) to perform a memory access operation, which includes a requested memory address. A determination is made as to whether a data block (e.g., page) associated with the requested memory address resides in a cache memory. When the data block associated with the requested memory address is not in the cache memory, a memory read-ahead process is performed which includes identifying a learned memory access pattern associated with the requestor, wherein the learned memory access pattern includes a plurality of data blocks starting with the data block associated with the requested memory address, and prefetching the plurality of data blocks associated with the learned memory access pattern into the cache memory.

Memory read-ahead using learned memory access patterns

Systems, methods, and articles of manufacture comprising processor-readable storage media are provided for implementing an accelerated data storage synchronization to protect data in a storage system. For example, a first server node accesses a data block from a protected storage system in response to a request from an application executing on the first server node. A modified data block is written to a write-back cache of the first server node. A synchronization operation is performed in response to a request by the application, which includes sending a copy of the modified data block in the write-back cache to a second server node to temporarily store the copy of the modified data block in a storage media of the second server node, and informing the application that the synchronization operation is deemed complete after the copy of the modified data block is stored in the storage media of the second server node.

Accelerated data storage synchronization for node fault protection in distributed storage system

Methods and apparatus for computer systems having first and second memory tier having regions, physical memory having page caches that are shareable with multiple ones of the regions in the first memory tier and the regions in the second memory tier, and virtual memory having mmaps of ones of the regions in the first memory tier and ones of the regions in the second memory tier, wherein the mmaps are associated with multiple ones of the pages caches.

Methods and apparatus for multiple memory maps and multiple page caches in tiered memory

Systems and methods are provided for implementing accelerated data access operations. For example, a method for accessing data includes loading an image into a memory, wherein the image comprises a plurality of image layers, and wherein each image layer references one or more data blocks. A request is received for a target data block from an active process operating on the image. Data structures associated with the target data bock are then accessed, wherein the data structures comprise metadata of a previous lookup path that was performed across one or more of the image layers of the image to access the target data block. The metadata of the data structures is utilized to access an existing in-memory copy of the target data block without having to traverse the previous lookup path, and the in-memory copy of target data block is returned to the active process.

Accelerated data access operations

Methods and apparatus for providing region zero-fill on demand for tiered memory including a first region in a first memory tier having a page cache in physical memory, where virtual memory includes a mmap of the first region. An input can be controlled between zeroes and the first region to the page cache.

Methods and apparatus for memory tier page cache with zero file

Data backup systems and methods are provided to asynchronously mirror in-memory checkpoint images generated by a process executing on a host system. For example, a data backup system is configured to: receive a checkpoint request message to store a copy of a checkpoint image which includes memory pages residing in a first memory region of memory on the host system; pull a copy of metadata associated with the memory pages from a second memory region of memory on the host system; pull a copy of the memory pages of the checkpoint image from the first memory region; generate a copy of the checkpoint image using the memory pages and associated metadata pulled from the host system; and store the copy of the checkpoint image in a storage device with an in-memory format that is the same as the checkpoint image residing in the first memory region on the host system.

Asynchronous in-memory data backup system

At least one memory region from a memory device of a host computing system is identified as containing data considered to be inactive. The host computing system is operatively coupled to a storage system external to the host computing system and configured to execute an application program. At least one memory region is stored as an archived memory region on a non-volatile storage device of the storage system in a memory format utilized on the host computing system.

Archiving memory in memory centric architecture

A method, system and program product for enabling migration of Virtual Machines with concurrent access to data across two geographically disperse sites to enable load balancing across the two geographically disperse sites, by presenting over a network a read writable logical volume at a first site, presenting over a network a read writable logical volume at a second geographically disparate site; wherein the first volume and the second volume are configured to contain the same information, and enabling read write access to the volume at the first site or the volume at the second site for a first virtual machine while keeping the data consistent between the two sites to enable transparent migration of the virtual machine to load balancing across the two sites according to at least one load balancing metric.

Non-disruptive load-balancing of virtual machines between data centers

Systems, methods, and articles of manufacture comprising processor-readable storage media are provided for managing memory by persistence of page access heuristics. For example, a method includes collecting, during run-time execution of an application, access heuristics with regard to each page of data that is accessed by the application and cached in a page cache associated with the application, and generating metadata for each cached page in the page cache. The metadata for a given cached page represents the access heuristics of the given cached page. The metadata for each cached page is maintained in a volatile memory during run-time execution of the application. Moreover, the metadata for each cached page is persistently stored in a non-volatile memory during run-time execution of the application. The persistently stored metadata for each cached page is accessed when the application is restarted to restore the page cache associated with the application to a previous state.

Growing community of inventors

Franklin, MA, United States of America

Kenneth J Taylor