Emc IP Holding Company LLC

Dell Products Inc.

Emc Corporation

Jonathan I Krasner

Steven R Chalmer

Chakib Ouarraoui

Serge Joseph Pirotte

Sweetesh Singh

Steven T McClure

Peter Yale Kushner

Ro Monserrat

Daryl Francis Kinney

Jerome J Cartmell

Ian Wigmore

Philip E Tamer

Jason J Duquette

William Alan Hatstat

Matt Fredette

Thomas Mackintosh

Arieh Don

Jean-Pierre Bono

Adnan Sahin

Michael J Scharland

Malak Alshawabkeh

Stephen Smaldone

John T Fitzgerald

James Marriot Guyer

Scott Rowlands

Preston F Crow

Matthew H Fredette

Julie Zhivich

Steve Chalmer

Alexander Veprinsky

James H Torrey

Bruce Ferjulian

Velmurugan Rathnam

Li Lang

Avraham Goldin

Rocio Aldrich

Steve McClure

Chakib Ourraoui

Kenneth Vineleaf

John D Sullivan

A system can maintain a group of data processing units and a storage array that comprises a group of sub-logical unit numbers of storage. The system can collect, by a central processing unit, first data indicative of input and output events for the storage array. The system can process, by respective data processing units, respective autoregressive integrated moving average models for respective sub-logical unit numbers of the group of sub-logical unit numbers with the first data, to generate respective statuses that indicate respective frequencies of access of the respective sub-logical unit numbers. The system can determine, by the central processing unit, respective classifications for respective sub-logical unit numbers of the group of sub-logical unit numbers of storage based on the respective statuses. The system can compress, by a compression engine, second data stored in at least some of the respective sub-logical unit numbers based on the respective classifications.

Offloading machine learning capabilities

A method for use in a computing device, the method comprising: detecting a request to transmit data, the data including a plurality of data portions, each of the data portions being associated with a respective remote device that the data portion is destined for; identifying a list of addresses that is associated with the request, each of the addresses in the list corresponding to the respective remote device of a different one of the plurality of data portions; detecting whether the request is local; when the request is not local, transmitting the data over a communications network; when the request is local transmitting each of the data portions in the plurality over the respective DMA link that couples the computing device with the data portion's respective remote device.

Efficient resource selection in a hybrid fabric and NTB-based storage system

A pair of compute nodes, each having a separate PCIe root complex, are interconnected by a PCIe Non-Transparent Bridge (NTB). An instance of a NTB monitoring process is started for each root complex, and the CPU affinity of the NTB monitoring processes are set to cause each NTB monitoring process to be executed on CPU resources of each respective CPU root complex. The NTB monitoring process on a given root complex is allowed to sleep until a triggering event occurs that causes the NTB monitoring process to wake and determine the state of the NTB. One such triggering event is a failure of an atomicity algorithm on the compute node to obtain a lock on peer memory in connection with implementing an atomic read operation on the peer memory over the NTB.

Method and apparatus for monitoring a PCIe NTB

A deterministic hardware indictment process is used to indict one Compute Node (CN) of a pair of CNs in response to occurrence of a fatal error on a Peripheral Component Interconnect Express (PCIe) Non-Transparent Bridge (NTB) interconnecting the two CN. Status information is exchanged between the CNs on a communication medium that is separate from the NTB. The indictment process is run locally on each CN based on whatever information is available to the CN. When the CN does not receive status information from the other CN, the CN runs a self-indictment check. When status information is received, and only one CN reports an error, the indictment process indicts the compute node that reported the error. If both CNs report errors, an error severity comparison process is used to select a CN to be indicted. If the reported errors are equally severe, a default CN is indicted.

Deterministic hardware indictment in response to fatal PCIe ntb error

A storage engine has a pair of compute nodes, each compute node having a separate PCIe root complex and attached memory. The PCIe root complexes are interconnected by multiple Non-Transparent Bridge (NTB) links. The NTB resources are unequally shared, such that host IO devices are required to use a first subset of the NTB links to implement memory access operations on the memory of the peer compute node, whereas storage software memory access operations are able to be implemented on all of the NTB links. A NTB link arbitration system arbitrates usage of the first and second subsets of NTB links by the storage software, to distribute subsets of the storage software memory access operations on peer memory to the first and second subsets of NTB links, while causing all host IO device memory access operations on peer memory to be implemented on the first set of NTB links.

Non-transparent bridge selection

Disclosed are techniques for compressing data in a data storage system comprising searching a cluster of nearest neighbors, wherein the cluster has been created using a locality sensitive hashing algorithm, to determine if a data block can be compressed. In alternate embodiments, nearest neighbor clusters can be formed using unsupervised learning. Additionally, nearest neighbors can also be formed in alternate embodiments using one or more of the following algorithms: a k-means clustering algorithm, a k-medoids clustering algorithm, a mean shift algorithm, a generalized method of moment (GMM) algorithm, or a density based spatial clustering of applications with noise (DBSCAN) algorithm.

Data compression using nearest neighbor cluster

A host IO devices directly implements host read operations on both local memory, and on peer memory via a PCIe non-transparent bridge. When a host read operation is received by a host IO device from a host, the host IO device uses an API to obtain the physical address of the requested data on the peer memory, and generates a PCIe Transaction Layer Packet (TLP) addressed to the address in the peer memory. The TLP addressed to an address in the peer memory is passed over the NTB to the peer compute node to retrieve the data stored in the addressed slot of peer memory. The requested data is returned to the host IO device over the NTB, stored in a buffer, and read out to the host to directy respond to the host read operation.

Host IO device direct read operations on peer memory over a PCIe non-transparent bridge

A data processing device for providing data storage services includes memory, persistent storage, and a device manager. The persistent storage stores a code repository. The state manager obtains a code update request for a new state for the device manager; injects state manager code to begin execution of the state manager in response to obtaining the code update request; updates the device manager data using the code repository and a memory region state map to obtain an updated device manager; and extracts the state manager code to resume execution of the updated device manager.

System and method for state management

Techniques for allocating resources in a system may include: monitoring, using a first proportional-integral-derivative (PID) controller, a size of a pool of free shared resources of a first type; responsive to determining the size of the pool of the free shared resources is at least a minimum threshold, providing the size of the pool of free shared resources as an input to a second PID controller; monitoring, using the second PID controller, a total amount of resources of the first type that are available; determining, using the second PID controller and in accordance with one or more resource policies for one or more applications, a deallocation rate or amount; deallocating, using the second PID controller and in accordance with the deallocation rate or amount, resources of the first type; and allocating a least a first of the deallocated resources for use by one of the applications.

Resource monitoring and allocation using proportional-integral-derivative controllers

A method for managing data includes obtaining, by a hypervisor on a host computing device, a write request for storing first data, and in response to the write request: identifying a first set of memory segments associated with the first data using a memory mapper, wherein the first set of memory segments is associated with a memory device, making a first determination that the memory device is local to the host computing device, and in response to the first determination: storing the first data in the first set of memory segments.

Method and system for providing processor-addressable persistent memory to guest operating systems in a storage system

Techniques for data deduplication may include: receiving write operations that write first data; partitioning the first data into a plurality of data portions; generating, using a first hash function, a plurality of data deduplication hash values for the plurality of data portions, wherein a first data deduplication hash value of the plurality of data deduplication hash values is produced by said generating for a first of the plurality of data portions; performing first processing using a Bloom filter to determine whether the first data deduplication hash value has a corresponding first entry in a data store of deduplication hash values; and responsive to the first processing determining the first data deduplication hash value does not have the corresponding first entry in the data store of deduplication hash values, performing second processing, said second processing including adding the corresponding first entry in the data store of deduplication hash values.

Data deduplication acceleration

A method for processing data includes receiving an offload request by a first virtual machine (VM), issuing, in response to the offload request and based on a processing pipeline, a processing request to a processing unit, and servicing, by the processing unit, the processing request to obtain a result.

Method and system for processing data using a processing pipeline and processing units

In general, in one aspect, the invention relates to a method for processing data, the method includes obtaining, by a host operating system (OS) on a host computing device, a notification of a power down. The further includes, in response to the notification, performing a data storage analysis on data stored in host OS memory to identify a plurality of processing tasks to perform on the data, making a first determination, based on the data storage analysis, that data processing is to be offloaded to a graphics processing unit, and in response to the first determination: sending a data processing request to the processing unit, obtaining a second notification associated with processed data from the graphics processing unit, and storing the processed data in a backup storage device.

Method and system for optimizing a host computing device power down through offload capabilities

A method for managing data includes generating, by an offload device, predicted active logical partition data using an active logical partition mapping obtained from a host computing device, generating logical partition correlation data using active memory track maps obtained from the host computing device, generating most probable tracks using the predicted active logical partition data and the logical partition correlation data, and sending the most probable tracks to the host computing device, wherein the host computing device evicts data from a memory device based on the most probable tracks.

Method and system for efficiently evicting data from memory slots in a storage system

A method for processing data includes receiving a write request by a host operating system during a predetermined time window, storing data associated with the write request in a shared memory, making a first determination that a threshold number of write requests are obtained within the predetermined time window, and, in response to the first determination, sending an offload request to a virtual machine (VM), wherein the offload request specifies at least the write request.

Growing community of inventors

Coventry, RI, United States of America

Jonathan I Krasner