Vmware, Inc.

Edward Joseph Goggin

Hariharan Subramanian

Sanjana Aswani

Rupesh Bajaj

Vibhor Arunkumar Patale

Kiran R Joshi

Jorge Guerra Delgado

Srinath Premachandran

Sandeep Rangaswamy

Gopala Suryanarayana

Anurekh Saxena

Radhika Vullikanti

Adam M Belay

Wenguang Wang

Eric Shannon Knauft

Jin Zhang

Radu Berinde

Ruocheng Li

In accordance with the present disclosure, files may be deduplicated in a distributed storage system having a plurality of storage volumes. A uniqueness metric for each file may indicate a degree of deduplication of the respective data files in the given storage volume. The uniqueness metric may be used to identify files for rebalancing in the distributed storage system. The uniqueness metric may be efficiently calculated with enough accuracy using a sampling methodology.

File granular data de-duplication effectiveness metric for data de-duplication

Exemplary methods, apparatuses, and systems determine that a disk drive in a distributed storage system is failing. In response to determining that the disk drive is failing, it is determined if an object has object components stored on the failing disk drive. Object components are flagged based on whether the object components will become inaccessible in the event the disk drive fails or is unmounted. Object components that are flagged with an evacuate flag are migrated from the failing disk drive to one or more other disk drives in the distributed storage system to prevent objects from becoming inaccessible.

Automatically removing dependency on slow disks in a distributed storage system

Techniques for implementing data deduplication in conjunction with thick and thin provisioning of storage objects are provided. In one embodiment, a system can receive a write request directed to a storage object stored by the system and can determine whether the storage object is a thin or thick object. If the storage object is a thin object, the system can calculate a usage value by adding a total amount of physical storage space used in the system to a total amount of storage space reserved for thick storage objects in the system and further subtracting a total amount of reserved storage space for the thick storage objects that are filled with unique data. The system can then reject the write request if the usage value is not less than the total storage capacity of the system.

Data deduplication with support for both thick and thin provisioning of storage objects

A method is provided for use in a system that includes a host computing machine configured to implement a virtualization intermediary and that includes a physical storage adapter, the configures a virtual machine (VM) and a virtual function (VF) to support IO operations to physical storage through a direct IOV path to the VF of the physical storage adapter, the method comprises: creating by the virtualization intermediary mapping information that includes a first mapping between virtual disks and physical regions of physical storage and that includes a second mapping between virtual disks and virtual disk addresses; transmitting the mapping information from the virtualization intermediary over the physical storage adapter from a physical function (PF) of the physical storage adapter to the VF; associating a virtual port with the mapping information within the virtualization intermediary; binding the virtual port to the VF; communicating virtual disk addresses indicated within the second mapping within the transmitted mapping information for the allocated virtual disks to the VM.

Configuring VM and IO storage adapter VF for virtual target addressing during direct data access

A method is provided to migrate a virtual machine from a source computing machine to a destination computing machine comprising: suspending transmission of requests from a request queue disposed in source computing machine memory associated with the VM from the request queue to a VF; while suspending the transmission of requests, determining when no more outstanding responses to prior requests remain to be received; in response to a determination that no more outstanding responses to prior requests remain to be received, transferring state information that is indicative of locations of requests inserted to the request queue from the VF to a PF and from the PF to a memory region associated with a virtualization intermediary of the source computing machine. After transferring the state information to source computing machine memory associated with a virtualization intermediary, resuming transmission of requests from locations of the request queue indicated by the state information to the PF; and transmitting the requests from the PF to the physical storage.

Live migration of virtual machine during direct access to storage over SR IOV adapter

A method of direct access by a virtual machine (VM) running on a host machine to physical storage via a virtual function (VF) running on an input/output (IO) adapter comprising: providing by a virtualization intermediary running on the host machine an indication of an active path associated with a virtual storage device; obtaining by a guest driver running within a guest operating system of the VM the stored indication of the active path from the shared memory region; dispatching an IO request by the guest driver to the VF that includes an indication of the active path; and sending by the VF an IO request that includes the indicated active path.

Virtualization intermediary/virtual machine guest operating system collaborative SCSI path management

Method for configuring a physical adapter with virtual function (VF) and physical function (PF) for controlling address translation between virtual disks and physical storage regions

A method is described for coalescing input/output (IO) interrupts to a virtual machine (VM) running on a host computer. At a virtualization layer of the host computer that supports execution of the VM receives an IO interrupt in response to a completion of an IO request of the VM, wherein a transmission of the IO request by the VM to an IO device bypasses the virtualization layer. The virtualization layer then determines whether the VM has responded to a most recently delivered IO interrupt to the VM by the virtualization layer, and drops the IO interrupt if the VM has not responded to the most recently delivered IO interrupt, thereby failing to deliver the IO interrupt to the VM.

Guest/hypervisor interrupt coalescing for storage adapter virtual function in guest passthrough mode

A method is provided for use in a system that includes a host machine that includes multiple physical CPUs (PCPUs) and at least two cache nodes that are shared by different sets of the PCPUs, comprising: creating in a memory device multiple sets of lanes each lane set associated with a respective PCPU set; tracking levels of processing activity of the PCPUs of each PCPU set; using an MSIX vector value to associate lanes with PCPUs; receiving a IO request from any given PCPU from among the multiple PCPUs; and assigning the IO request to a respective lane based at least in part upon the PCPU set associated with the lane and PCPU processing activity levels.

Virtual machine access to storage via a multi-queue IO storage adapter with optimized cache affinity and PCPU load balancing

A system and method are provided that involve a host computing machine and an SR IOV storage adapter in which the host machine hosts a virtual machine having a guest operating system (guest) coupled for direct passthrough IOV data path and also hosts a virtualization intermediary; a guest operating system (guest) and a virtualization intermediary exchange information concerning IO completions through a shared memory space; the guest writes information to a shared memory space that is indicative of whether an IO completion queue has reached a fill level since the most recently dispatched interrupt at which it is unsafe to coalesce an interrupt; the virtualization intermediary writes information to the shared memory space that is indicative of the interrupt most recently delivered to the guest; the virtualization intermediary reads the information written by the guest to the shared memory space that is indicative of whether an IO completion queue has reached a fill level since the most recently dispatched interrupt at which it is unsafe to coalesce an interrupt; the guest reads the information written to the shared memory space that is indicative of the interrupt most recently delivered to the guest; virtual interrupt coalescing determinations are made based upon the information shred through the shared memory space.

A method of virtual machine (VM) access to physical storage through a direct path to a virtual function (VF) of a storage adapter, the method for use in a system that includes a host computing machine configured to implement a virtualization intermediary and the virtual machine (VM) and that includes the storage adapter the method, comprising: sending virtual SCSI IO request from the VM to the physical storage that identifies a virtual disk address; mapping within the VF the identified virtual address to at least one physical region of the physical storage; creating within the VF a physical SCSI IO request that identifies a physical address for the mapped-to physical region; sending the physical SCSI IO request from the VF to the physical storage.

Growing community of inventors

Concord, MA, United States of America

Edward Joseph Goggin