Emc Corporation

Emc IP Holding Company LLC

Los Alamos National Security, LLC

Digital Equipment Corporation

Triad National Security, LLC

Emc IP Holding Corporation LLC

Percy Tzelnic

Sorin Faibish

John Michael Bent

Uday K Gupta

Uresh K Vahalia

James M Pedone, Jr

Gary Alan Grider

Dennis P Ting

Dennis P J Ting

Betti Porat

Natan Vishlitzky

Aaron Torres

Steven M Blumenau

Xiaoye Jiang

Erez Ofer

John T Fitzgerald

John Forecast

Lev Vaitzblit

Mark C Lippitt

Gang Ma

James M McGillis

Scott R Dunham

Dominique P Cote

Mark A Kaufman

David Bonnie

Hsing-bung Chen

Zhenhua Zhang

Wayne William Duso

Peter C Bixby

Kadangode K Ramakrishnan

James J Moore

Jingwang Zhang

Robert Wilson

Aju John

Dennis Pei Jean Ting

Sasan Teymouri

Yikang Xu

Kannan M Alagappan

Boyd Wilson

Latchesar A Ionkov

Adam Manzanares

A multi-tier storage system is provided with direct client access to an archive storage tier for input/output operations. An exemplary method comprises communicating over a network with (i) a cluster file system on a first storage tier, and (ii) a second archive storage tier comprising an object store; providing a client of the cluster file system with access to one or more files in the cluster file system on the first storage tier; and executing a translation shim to provide the client of the cluster file system with one or more of read and write access to one or more files on the second archive storage tier. The translation shim converts between the protocols of the cluster file system and the protocols of the second archive storage tier, to allow unmodified applications to optionally access the second archive storage tier using existing cluster file system protocols.

Multi-tier storage system with direct client access to archive storage tier

Example embodiments of the present invention relate to a method, a system, and a computer program product for maintaining dynamically loadable namespaces. The method includes presenting, to an application, a file system having a global single namespace (SN) comprising a plurality of dynamically-loadable namespaces (DLNs) maintained in a capacity tier (CT) of object storage of a storage system. In response to a job submitted by the application, dynamically loading a first dynamically-loadable namespace, including the metadata only for the first dynamically-loadable namespace, associated with the job from the capacity tier of object storage into a fast tier (FT) of storage of the storage system.

Method and system for reducing metadata overhead in a two-tier storage architecture

An apparatus in one embodiment comprises a storage system having at least first and second storage tiers each comprising a plurality of storage devices. The storage system implements a parallel file system comprising at least first and second sets of storage servers. The first set of storage servers is configured to access directories of the parallel file system that are stored in storage devices of the first storage tier, and the second set of storage servers is configured to access directories of the parallel file system that are stored in storage devices of the second storage tier. The storage system is further configured to move data between the first and second storage tiers by moving data files between different directories of the respective first and second storage tiers. One or more servers of the first set may be generated as respective new server instances on one or more compute nodes.

Storage system with directory-based storage tiering

A computer-executable method, system, and computer program product for managing a cloud computing resource in communication with a data storage system, the computer-executable method, system, and computer program product comprising receiving configuration information at the data storage system related to a host application; determining an optimized configuration for the host application; and creating an implementation of the optimized configuration based on one or more resources enabled to be accessed by the data storage system.

Method for running HPC jobs

A storage system comprises a shared storage environment that includes a storage array having at least one storage volume shared between first and second host devices. The storage system further comprises a server associated with the storage array, at least first and second clients associated with the respective first and second host devices, and a first block cache arranged between the first client and the storage array. The server is configured to coordinate operations of the first and second clients relating to the storage volume shared between the first and second host devices in a manner that ensures coherency of data stored in the first block cache. The server may comprise a storage block mapping protocol (SBMP) server and the first and second clients may comprise respective SBMP clients. The block cache is illustratively implemented using a VFCache or other type of server flash cache.

Block caching in a shared storage environment

Techniques are provided for scheduling computational tasks among multiple classes of storage resources based on a job classification. A job to be executed is classified into one of a plurality of predefined job classes. Each predefined job class is associated with a corresponding one of a plurality of predefined storage classes. The job is then assigned based on the classification to one of the storage resources of the predefined storage class associated with the classified predefined job class. Exemplary predefined storage classes include a performance class, a capacity class, a key-value storage class, and a shingled disk drive class. Exemplary predefined job classes include a CPU Intensive job class, an IO Intensive job class and a Small IO job class. Data required for a job is optionally prefetched before the job is assigned to a storage device. Data objects to be evicted from a storage device are optionally selected based on an anticipated future access.

Scheduling computational tasks among multiple classes of storage resources based on job classification

Techniques are provided for encryption and decryption of time series data using a digital filter array. A plurality of digital time series samples generated by a digital time series generation device are applied to a digital filter array (e.g., a finite impulse response filter or an infinite impulse response filter) that combines delayed versions of the digital time series samples using a plurality of coefficients to generate a plurality of encrypted digital time series samples. The plurality of coefficients comprise a portion of an encryption key associated with the digital time series generation device. The digital time series generation device has an associated device identifier, and the device identifier and the corresponding encryption key are registered with a server. The plurality of encrypted digital time series samples and the device identifier are provided to the server for decryption using an inverse digital filter array based on the encryption key.

Encryption and decryption of time series data using a digital filter array

An apparatus in one embodiment comprises a storage system configured to implement at least one scalable video server. The storage system comprises a software-defined storage pool, and the scalable video server comprises a plurality of file system storage nodes each including a corresponding portion of the software-defined storage pool and an associated file system server. A streaming bandwidth of the scalable video server for a given video stream is controlled by adjusting the number of file system storage nodes utilized for the given video stream in the scalable video server. The file system servers of the respective file system storage nodes are configured to interact with a file system client associated with the given video stream. The streaming bandwidth of the scalable video server for the given video stream may be dynamically adjusted by adding or deleting file system storage nodes to or from the scalable video server.

Storage system for scalable video streaming using software-defined storage pool

An apparatus in one embodiment comprises a reconfigurable multi-tier storage system having at least a front-end storage tier and a back-end storage tier. The multi-tier storage system is implemented at least in part utilizing a plurality of replicated storage units. One or more of the replicated storage units are each utilized in a first configuration of the reconfigurable multi-tier storage system to implement a portion of at least one of a burst buffer appliance and a software-defined storage pool of the front-end storage tier. At least one of the one or more replicated storage units is utilized in a second configuration of the reconfigurable multi-tier storage system to implement a portion of at least one of a scale-out network-attached storage cluster and an object store of the back-end storage tier. A given one of the replicated storage units may be reconfigured by rebooting it using different software modules.

Reconfigurable multi-tier storage system comprising replicated storage units

An apparatus in one embodiment comprises a multi-tier storage system having at least a front-end storage tier and a back-end storage tier. The multi-tier storage system is implemented at least in part utilizing a plurality of virtual machines of cloud infrastructure. The front-end storage tier comprises a plurality of storage nodes of a cluster file system, with the storage nodes being implemented on respective ones of the virtual machines. The front-end storage tier further comprises a software-defined storage pool accessible to the storage nodes and implemented utilizing local disk resources of respective ones of the virtual machines. The back-end storage tier of the multi-tier storage system comprises at least one object store. At least a subset of the virtual machines may further comprise respective compute nodes configured to access the multi-tier storage system. Other illustrative embodiments include systems, methods and processor-readable storage media.

Storage system comprising cluster file system storage nodes and software-defined storage pool in cloud infrastructure

Metadata associated with a plurality of sub-files associated with a single shared file is stored in a parallel file system. A plurality of processes generate a shared file. A compute node implements a Parallel Log Structured File System (PLFS) library to store at least one portion of the shared file and metadata for the at least one portion of the shared file on one or more of the plurality of object storage servers. The compute node is further configured to store the metadata by striping the metadata across a plurality of subdirectories of the shared file. The metadata is optionally striped across the plurality of subdirectories in a round-robin manner. The plurality of subdirectories are stored on one or more of the object storage servers. Write and read processes optionally communicate using a message passing interface. A given write process optionally writes metadata for a given portion of the shared file to an index file in a particular one of the subdirectories corresponding to the given portion.

Parallel file system with striped metadata

An apparatus in one embodiment comprises a storage system having at least first and second storage tiers each comprising a plurality of storage devices. The storage system implements at least one configurable data mover module adapted to interact with the first and second storage tiers and comprising at least one power management configuration file. The data mover module alters a manner in which data is moved between the first and second storage tiers based at least in part on a change in state of the power management configuration file. For example, the data mover module may be configurable via the power management configuration file in a mode of operation in which data movement from the first storage tier to the second storage tier occurs for data files utilized by a given application only after completion of a compute phase for that application.

Multi-tier storage system with dynamic power management utilizing configurable data mover modules

An apparatus comprises a multi-tier storage system having a front-end storage tier, a back-end storage tier and a data mover module coupled to the front-end and back-end storage tiers and configured to control movement of data between the storage tiers. A translator at least partially incorporated in or otherwise associated with the multi-tier storage system implements a plurality of translation services and one or more index tables. A given one of the translation services is configured to access at least one of the index tables in order to determine a dynamically loadable namespace identifier associated with an application running on an analysis node that communicates with the multi-tier storage system over at least one network, and to instantiate a name node corresponding to the dynamically loadable namespace identifier. Particular objects of an object store of the back-end storage tier are made accessible to the application via the name node.

Translation-based name node configuration for object access in a multi-tier storage system

A multi-tier storage system is provided that employs a burst buffer middleware appliance for batch messaging. An exemplary multi-tier storage system comprises at least first and second storage tiers comprising storage for storing data; and at least one burst buffer appliance, wherein the at least one burst buffer appliance comprises an abstract storage interface comprising a software interface module that communicates with at least one persistent storage device in at least one of the first and second storage tiers, wherein the at least one persistent storage device provides a list application programming interface, wherein the software interface module provides a batch operation comprised of a plurality of individual operations to the at least one persistent storage device using an aggregated message. The batch operation optionally comprises a batch operation for processing metadata. A burst buffer appliance is also provided for implementing the batch messaging techniques described herein.

Multi-tier storage environment with burst buffer middleware appliance for batch messaging

Techniques are provided for content-aware storage tiering techniques within a job scheduling system. A data node in a job scheduling environment receives at least one job to be executed for a portion of a larger logical file, and notifies one or more sibling data nodes to prefetch one or more related portions of the larger logical file. A write operation performed by an application client to store data on the data node is optionally intercepted at the data node and the associated data is stored in a performance storage tier of the data node. The data is optionally moved from the performance storage tier to a capacity storage tier if one or more predefined eviction criteria are satisfied. Data objects to be evicted are selected based on an anticipated future access of the data objects. The eviction selection is optionally based on grey box knowledge of the application client and/or information obtained from the application client.

Content-aware storage tiering techniques within a job scheduling system

Growing community of inventors

Concord, MA, United States of America

Percy Tzelnic