Emc IP Holding Company LLC

Dell Products Inc.

Emc Corporation

Scott Rowlands

Arieh Don

Erik P Smith

Igor Fradkin

Vinay G Rao

Krishna Deepak Nuthakki

Alan Rajapa

Stephen Richard Ives

James Marriot Guyer

Owen Crowley

Seema G Pai

Ningdong Li

Sanjib Mallick

Michael J Scharland

Jonathan I Krasner

Subin George

Kevin Martin

Julie Zhivich

James L Davidson

Jerome J Cartmell

Ro Monserrat

Brian Riordan

Vincent H Westin

Ramprasad Shetty

Anthony D Fong

Thomas Mackintosh

Daniel J Rodrigues

David Brown

A storage array is configured to detect single points of failure (SPoFs) that could cause a host application to fail. Zoning information obtained from network switches is used with initiator-target login information to determine whether all host server HBA port initiators logged in to a storage object maintained by the storage array are from the same zone. If so, a path-based SPOF alert is generated. If all host server HBA port initiators logged in to the storage object are from the same HBA, then a NIC-based SPOF alert is generated. A masking view is used with the zoning information and initiator-target login information to determine whether all host server HBA port initiators logged in to the storage object are from the same host server. If so, a clustering-based, nodal SPOF alert is generated.

SAN single path point of failure detection

At least one processing device is configured to control delivery of input-output (IO) operations from a host device to a storage system over selected ones of a plurality of paths through a network. The at least one processing device is further configured to designate one or more of the paths as being associated with a link performance issue, to temporarily suspend utilization of the one or more designated paths for delivery of IO operations from the host device to the storage system, to detect a configuration change that is indicative of potential resolution of the link performance issue, and to resume utilization of the one or more designated paths responsive to the detected configuration change. The at least one processing device illustratively comprises a multi-path input-output (MPIO) driver of the host device.

Multi-path layer configured with enhanced awareness of link performance issue resolution

Target port resets sent from a host server are used to identify HBA faults. A storage array monitors receipt of resets and calculates reset counts for each HBA port of each host server. If there are more than X resets within Y seconds received by a storage array port from a particular HBA port, that HBA port is identified as faulty if the storage array port has not received resets from other HBAs or other ports of the same HBA. If the storage array port has received resets from other ports of the same HBA but not from other HBAs, then the HBA is identified as faulty. If the initiators are grouped, both the HBA port and HBA are identified as potentially faulty. If none of the HBAs or HBA ports are identified as faulty, the cause of the resets is localized to the storage array, inter-switch links, or congestion.

Identifying host bus adapater faults

A host IO devices directly implements host read operations on both local memory, and on peer memory via a PCIe non-transparent bridge. When a host read operation is received by a host IO device from a host, the host IO device uses an API to obtain the physical address of the requested data on the peer memory, and generates a PCIe Transaction Layer Packet (TLP) addressed to the address in the peer memory. The TLP addressed to an address in the peer memory is passed over the NTB to the peer compute node to retrieve the data stored in the addressed slot of peer memory. The requested data is returned to the host IO device over the NTB, stored in a buffer, and read out to the host to directy respond to the host read operation.

Host IO device direct read operations on peer memory over a PCIe non-transparent bridge

A congestion and failure detector and localizer running on a storage array locally monitors ports of the storage array for IO failures and local link errors and remotely monitors ports of host initiators and host-side and storage array-side switches for link errors. Based on whether the local link error rate is increasing at any ports, whether IO failures are associated with a single host initiator port, and whether link error rate is increasing on both the host initiator and initiator-side switch, the congestion and failure detector and localizer generates a flag indicating either a physical link problem between the storage array and adjacent switch, ISL physical issue or spreading congestion, host initiator-side physical link problem, or path congestion. The flag identifies the storage array port, issue type, link, fabric name, and host initiator.

Discovering host-switch link and ISL issues from the storage array

A method, comprising: receiving a plurality of first response times, each of the plurality of first response times corresponding to a different one of a plurality of first network paths between a first host device and the first storage array, the plurality of first response times being received from the first host device; receiving a plurality of second response times, each of the plurality of second response times corresponding to a different one of a plurality of second network paths between a second host device and the first storage array, the plurality of second response times being received from the second host device; and processing the plurality of first response times and the plurality of second response times, wherein the plurality of first network paths and the plurality of second network paths are part of a communications network.

Method and apparatus for detecting degraded network performance

Bandwidth consumption for IO paths between a storage system and host may be managed. It may be determined whether there is congestion on a front-end port (FEP) link. For example, the storage system may monitor for a notification from the switch in accordance with a Fibre Channel (FC) protocol. If a notification is received indicating congestion on an FEP link, the bandwidth thresholds (BWTs) for one or more IO paths between the storage system and one or more hosts that include the FEP link may be reduced. The host port BWTs may continue to be reduced until a congestion notification communication has not been received for a predetermined amount of time, in response to which the host port BWTs for one or more host port links on IO paths that include the FEP link may be increased. Similar techniques may be employed for an FEP link determined to be faulty.

Managing IO path bandwidth

Embodiments of the present disclosure relate to traffic class management of NVMe (non-volatile memory express) traffic. One or more virtual controllers for at least one host adapter (HA) of a storage device are generated. Each virtual controller is assigned a unique controller identifier (ID) Additionally, one or more input/output (IO) queues for each virtual controller are established. Further, IO workloads are processed via each IO queue.

Hardware virtualization

Techniques for transmitting data may include: statically allocating a first buffer by a first entity; determining, by the second code entity, whether first data to be returned to the first entity fits into a data area of the first buffer; storing the first data in the data area of the first buffer responsive to determining the first data fits into the data area of the first buffer; and responsive to determining the first data does not fit into the data area of the first buffer, performing first processing by the second entity to store the first data in a dynamically allocated second buffer having a descriptor stored in the first buffer. The descriptor may include the address and size of the second buffer. The first and second entity may be code entities in the same system, or may be different systems that communicate using a remote direct memory access protocol.

Buffer allocation techniques

Information pertinent to a connectivity issue reported by a switch may be determined, compiled and reported; and action may be taken autonomously to address the connectivity issue based on the information. The additional information may be determined by the storage system, including querying a switch in response to the switch notifying the storage system of the I/O error, and the storage system accessing one or more data structures to determine additional information. The storage system may cause I/O communications on the I/O path to be at least temporarily diverted. For example, in response to receiving an I/O request on the I/O path corresponding to the connectivity issue, the storage system may send a communication to the host system that causes the host system not to send additional I/O requests on the I/O path, at least temporarily, as described in more detail elsewhere herein.

Managing I/O connectivity issues

Single points of failure (SPoFs) may be determined for I/O connectivity on a storage network. I/O path information may be determined for a storage device, for example, as a result of a host system logging into the storage network, and may be updated in response to events on the storage network. From this determined I/O path information, one or more SPoFs between a storage device and an application layer may be determined if, for the I/O path information collectively, it is determined that there is only one of any of the path components between the storage device and the application layer. The I/O path information may be displayed in a manner that facilitates a user identifying that there is an SPoF on an I/O path between a storage device and an application layer of a host system. Based on the determination of an SPoF, an alert may be issued.

Detecting single points of failure on a storage system

An illustrative embodiment includes a host device configured to communicate over a network with a storage system comprising a plurality of storage devices. The host device comprises a set of input-output queues and a multi-path input-output driver configured to select input-output operations from the set of input-output queues for delivery to the storage system over the network. The multi-path input-output driver is further configured to determine fabric identifiers for respective ones of a plurality of paths from the host device to the storage system, and to select particular ones of the paths for delivery of the input-output operations to the storage system based at least in part on the fabric identifiers. The fabric identifiers may be determined for the respective paths, for example, based at least in part on responses to a predetermined command sent over the paths by the multi-path input-output driver.

Host device with multi-path layer implementing path selection based at least in part on fabric identifiers

Communications between a host system and a storage system may be restricted to a single I/O path, at least temporarily, without manually manipulating physical connections to the host system and/or storage system or manually data structures defining I/O connectivity. In response to a communication to maintain only a first permitted I/O path between a host system and a storage system, I/O communications may be prevented on any I/O paths between the host system and the storage system, except for the first permitted I/O system, without modifying the definition of any permitted I/O paths on the storage system. One or more fields may be included in entries of a data structure that defines permitted I/O paths, the fields specifying whether an I/O path should be allowed to, or prevented from, carry communications between a host system and the storage system, at least temporarily.

Managing single path communication between a host and a storage system

Multiple virtual host ports corresponding to a same physical host port may be determined by or on behalf of a storage system, for example, in response to logging the one or more virtual host ports into the storage system. For one or more virtual host ports, it may be determined whether the virtual host port is connected to a same fabric port as another virtual host port, where a fabric port is a port of a fabric configured to connect to a virtual host port. If two virtual host ports are determined to be connected to a same fabric port, it may be concluded that the two virtual host ports correspond to (e.g., share) a same physical host port. One or more actions may be taken on a storage network based at least in part on a determination that two virtual host ports are sharing a same physical host port.

Determining multiple virtual host ports on a same physical host port

Techniques for processing I/O operations may include performing DMA (direct memory access) operations between a data storage system, one or more physical storage devices, and a hardware component that communicate over at least one bus using a DMA-based protocol, such as NVMe (Non-Volatile Memory Express). The hardware device may perform encryption and decryption processing of data that is, respectively, stored to, and read from, physical non-volatile storage. The hardware device may optionally perform other processing for other data-related operations such as any of data validation and integrity checking, data deduplication, data compression, and data decompression. When performing DMA transfers, multiple descriptors, such as SGLs (scatter gather lists) or PRPs (physical region pages), for multiple data portions having logically contiguous consecutive logical addresses may be combined into a single descriptor sent in a single DMA operation.

Growing community of inventors

Marietta, GA, United States of America

Scott Rowlands