Weka.io Ltd.

International Business Machines Corporation

Omri Palmon

Liran Zvibel

Maor Ben Dayan

Kanael Arditti

Tomer Filiba

Haim Helman

Shemer Schwarz

Ori Peleg

Kariel E Sandler

Artemy Voikhansky

Alex Goltman

Ofir Zohar

Shemer Schwartz

Ehood Garmiza

Efri Zeidner

Lior Segev

Maor Ben-Dayan

Methods and systems are provided for rapid failure recovery for a distributed storage system for failures by one or more nodes.

Methods and systems for rapid failure recovery for a distributed storage system

A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. The computing devices may push a point in time to a backend for a distributed storage system.

Pushing a point in time to a backend object storage for a distributed storage system

A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. A plurality of failure resilient stripes is distributed across the plurality of storage devices such that each of the plurality of failure resilient stripes spans a plurality of the storage devices. A graphics processing unit is operable to access data files from the failure resilient stripes, while bypassing a kernel page cache. Furthermore, these data files may be accessed in parallel by the graphics processing unit.

Resiliency schemes for distributed storage systems

A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. A plurality of failure resilient address spaces are distributed across the plurality of storage devices such that each of the plurality of failure resilient address spaces spans a plurality of the storage devices. The plurality of computing devices maintains metadata that maps each failure resilient address space to one of the plurality of computing devices. Each of the plurality of computing devices is operable to read from and write to a plurality of memory blocks, while maintaining an extent in metadata that maps the plurality of memory blocks to the failure resilient address space.

File operations in a distributed storage system

A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices comprises one or more of a plurality of storage devices. A plurality of failure resilient address spaces are distributed across the plurality of storage devices such that each of the plurality of failure resilient address spaces spans a plurality of the storage devices. Each one of the plurality of failure resilient address spaces is organized into a plurality of stripes. Each one or more stripes of the plurality of stripes is part of a respective one of a plurality of forward error correction (FEC) protection domains. Each of the plurality of stripes may comprise a plurality of storage blocks. Each block of a particular one of the plurality of stripes may reside on a different one of the plurality of storage devices.

Distributed erasure coded virtual file system

Methods and systems for electronic storage are provided. A storage system comprises a plurality of storage system front ends, a plurality of storage system back ends, and a plurality of solid state drive (SSD) agents. Each storage system front end resides on a server of a plurality of servers. Each server of the plurality of servers comprises one or more storage system back ends of the plurality of storage system back ends. Each storage system front end is able to receive I/O requests and relay information associated with the I/O requests to a relevant storage system back end. The relevant storage system back end communicates metadata associated with the I/O request to an SSD via an SSD agent.

Electronic storage system

A system comprises control circuitry that is operable to assign a first of a plurality of computing devices to serve file system requests destined for any of a first plurality of network addresses; assign a second of the computing devices to serve file system requests destined for any of a second plurality of network addresses; maintain statistics regarding file system requests sent to each of the first plurality of network addresses and the second plurality of network addresses; and reassign, based on the statistics, the first of the computing devices to serve file system requests destined for a selected one of the second plurality of network addresses.

Load balanced network file accesses

One or more computing devices may comprise congestion management circuitry, one or more client file system request buffers, and DESS interface circuitry. The one or more client file system request buffers is/are operable to queue first client file system requests of a first priority level and second client file system requests of a second priority level, where the first priority level is higher priority than the second priority level. The DESS interface circuitry is operable to determine a choking level according to the load on a plurality of DESS resources. Individual load values of the DESS resources are mapped to a composite load value using a first function. The composite load value is mapped to a congestion contribution using a second function. And, the congestion contribution is mapped to a choking level using a third function.

Quality of service management in a distributed storage system

A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. Data communication is made more efficient by removing the need to copy data in the networking stack, using hardware accelerated end-to-end checksum calculation, and supporting transmission formatting of data and header for special cases.

Efficient networking for a distributed storage system

A computing device comprising a frontend and a backend is operably coupled to a plurality of storage devices. The backend comprises a plurality of buckets. Each bucket is operable to build a failure-protected stipe that spans two or more of the plurality of the storage devices. The frontend is operable to encrypt data as it enters the plurality of storage devices and decrypt data as it leaves the plurality of storage devices.

Encryption for a distributed filesystem

A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. The computing devices may use local caches in a coherent manner when accessing the plurality of storage devices.

Implementing coherency and page cache support in a distributed way for files

A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. A plurality of failure resilient address spaces are distributed across the plurality of storage devices such that each of the plurality of failure resilient address spaces spans a plurality of the storage devices. The plurality of computing devices maintains metadata that maps each failure resilient address space to one of the plurality of computing devices. The metadata is grouped into buckets. Each bucket is stored in the backend of a computing device. Access to a storage device may be managed according to an availability of the storage device. Reads from and writes to a storage device may be redirected when, for example, the storage device is over-utilized or performing garbage collection.

Access redirection in a distributive file system

A plurality of computing devices are communicatively coupled to each other via a network, and each of the plurality of computing devices is operably coupled to one or more of a plurality of storage devices. A plurality of failure resilient address spaces are distributed across the plurality of storage devices such that each of the plurality of failure resilient address spaces spans a plurality of the storage devices. The plurality of computing devices maintains metadata that maps each failure resilient address space to one of the plurality of computing devices. The metadata is grouped into buckets. Each bucket is stored in a group of computing devices. However, only the leader of the group is able to directly access a particular bucket at any given time.

Metadata control in a load-balanced distributed storage system

A system comprises a plurality of computing devices that are communicatively coupled via a network and have a file system distributed among them, and comprises one or more file system request buffers residing on one or more of the plurality of computing devices. File system choking management circuitry that resides on one or more of the plurality of computing devices is operable to separately control: a first rate at which a first type of file system requests (e.g., one of data requests, data read requests, data write requests, metadata requests, metadata read requests, and metadata write requests) are fetched from the one or more buffers, and a second rate at which a second type of file system requests (e.g., another of data requests, data read requests, data write requests, metadata requests, metadata read requests, and metadata write requests) are fetched from the one or more buffers.

Growing community of inventors

Tel Aviv, Israel

Omri Palmon