Emc IP Holding Company LLC

Emc Corporation

Los Alamos National Security, LLC

Emc IP Holding Corporation LLC

James M Pedone, Jr

Sorin Faibish

John Michael Bent

Percy Tzelnic

Sassan Teymouri

Uday K Gupta

Dennis P Ting

Dominique P Cote

Philippe Armangau

Dennis P J Ting

Gang Ma

Eriks Sasha Paegle

Patrick S Combes

Gary Alan Grider

Tao Peng

James J Moore

Dennis Pei Jean Ting

Sasan Teymouri

Ivan Bassov

Xiaoye Jiang

Per W Brashers

Jason Glasgow

Ronald A Miller, Ii

Boyd Wilson

Latchesar A Ionkov

A method is used in analyzing a storage system using a machine learning system. Data gathered from information associated with operations performed in a storage system is analyzed. The storage system is comprised of a plurality of components. A bitmap image is created based on the gathered data, where at least one of the plurality of components is represented in the bitmap image. The machine learning system is trained using the bitmap image, where the bitmap image is organized to depict the plurality of components of the storage system.

Analyzing storage systems using machine learning systems

Techniques for detecting state changes in a system may include receiving a first neural network that is trained to detect when the system transitions into a first resulting state, wherein the system transitions into at least a first intermediate state prior to transitioning into the final resulting state; training the first neural network using a first plurality of inputs denoting the system in the first intermediate state; obtaining a plurality of sets of internal state information of the first neural network, each set of the plurality of sets denoting an internal state of the first neural network at a different point in time after the first neural network has processed at least a portion of the first plurality of inputs; and training a second neural network, using the plurality of sets of internal state information, to detect the first intermediate state.

Using machine learning to detect system changes

A method system, and computer program product for improving data reduction using aggregate machine learning systems comprising receiving, by an aggregating machine learning system from one or more machine learning systems associated with a set of one or more storage arrays, a first set of output parameters indicative of performance metrics for the set of the one or more storage arrays, aggregating, by the aggregating machine learning system, the first set of output parameters, resulting in a second set of output parameters, and sending, from the aggregating machine learning system, at least one member of the second set of output parameters as an input to at least one of the one or more machine learning systems associated with the set of the one or more storage arrays.

Data reduction improvement using aggregated machine learning

Methods and apparatus are provided for filtering dynamically loadable namespaces (DLNs). An exemplary method comprises, in response to a job submitted by an application, obtaining a DLN portion of a global single namespace of a file system, wherein the DLN is associated with the job and is maintained in a capacity tier of a storage system; obtaining filtering directives from a user; reducing the DLN using a filtering mechanism on a directory tree associated with the DLN, based on the filtering directives, by removing files in the directory tree of the DLN that do not satisfy requirements of the filtering directives to generate a filtered DLN; and dynamically loading the filtered DLN, including reduced metadata for the filtered DLN relative to the DLN, from the capacity tier into a performance tier of the storage system for processing by the application.

Methods and apparatus for filtering dynamically loadable namespaces (DLNs)

Methods and apparatus are provided for reorganizing dynamically loadable namespaces (DLNs). In one exemplary embodiment, a method comprises the steps of, in response to a job submitted by an application, obtaining a DLN portion of a global single namespace of a file system, wherein the DLN is associated with the job and is maintained in a capacity tier of object storage of a storage system; obtaining one or more reordering directives from a user; rearranging one or more files in the DLN into a new directory hierarchy based on the one or more reordering directives to generate a reordered DLN; and dynamically loading the reordered DLN, including the metadata only for the reordered DLN, from the capacity tier of object storage into a performance tier of storage of the storage system for processing by the application. The reordered DLN is merged into the DLN following one or more modifications to the reordered DLN.

Methods and apparatus for reorganizing dynamically loadable namespaces (DLNs)

A multi-tier storage system is provided with direct client access to an archive storage tier for input/output operations. An exemplary method comprises communicating over a network with (i) a cluster file system on a first storage tier, and (ii) a second archive storage tier comprising an object store; providing a client of the cluster file system with access to one or more files in the cluster file system on the first storage tier; and executing a translation shim to provide the client of the cluster file system with one or more of read and write access to one or more files on the second archive storage tier. The translation shim converts between the protocols of the cluster file system and the protocols of the second archive storage tier, to allow unmodified applications to optionally access the second archive storage tier using existing cluster file system protocols.

Multi-tier storage system with direct client access to archive storage tier

An apparatus in one embodiment comprises a multi-tier storage system having at least a front-end storage tier, a back-end storage tier and a data mover module configured to control movement of data between the storage tiers. The multi-tier storage system is configured to receive a plurality of files, such as relatively small size files containing data from respective Internet of Things (IoT) devices. The front-end storage tier comprises a file system and at least one key-value store. The multi-tier storage system is configured to combine multiple ones of the received files written to at least one specified directory of the file system into an object for storage in association with a key in the key-value store. The object comprises multiple sub-keys identifying respective ones of the multiple files within the object. The data mover module may be configured to copy the object from the key-value store to the directory.

Multi-tier storage system configured for efficient management of small files associated with Internet of Things

Techniques for determining data reduction options may include: receiving first data reduction information regarding compression and deduplication of chunks of a data set; determining, in accordance with the first data reduction information for the data set, first settings denoting whether compression and deduplication are enabled or disabled for the data set; receiving, during a first time period when the first settings are effective, writes directed to the data set; receiving second data reduction information regarding compression and deduplication of chunks of the data set modified by writes during the first time period; and determining, in accordance with the second plurality of data reduction statistics for the data set, second settings denoting whether compression and deduplication are enabled or disabled for the data set. Data reduction information about the data set may include histograms regarding compressed sizes and detected duplications and may include data reduction information regarding different data set regions.

Techniques for optimizing data reduction by understanding application data

An apparatus in one embodiment comprises a storage system having at least first and second storage tiers each comprising a plurality of storage devices. The storage system implements a parallel file system comprising at least first and second sets of storage servers. The first set of storage servers is configured to access directories of the parallel file system that are stored in storage devices of the first storage tier, and the second set of storage servers is configured to access directories of the parallel file system that are stored in storage devices of the second storage tier. The storage system is further configured to move data between the first and second storage tiers by moving data files between different directories of the respective first and second storage tiers. One or more servers of the first set may be generated as respective new server instances on one or more compute nodes.

Storage system with directory-based storage tiering

A System, Computer program product, and computer-executable method of authenticating a user to a remote computing network, the System, Computer program product, and computer-executable including receiving a first portion of authentication data at a server, wherein the first portion of authentication data includes at least one feature of an image and analyzing the received first portion of authentication data by comparing the first portion of authentication data to a second portion of authentication data stored at the server.

Age adapted biometric authentication

A computer-executable method, system, and computer program product for managing a cloud computing resource in communication with a data storage system, the computer-executable method, system, and computer program product comprising receiving configuration information at the data storage system related to a host application; determining an optimized configuration for the host application; and creating an implementation of the optimized configuration based on one or more resources enabled to be accessed by the data storage system.

Method for running HPC jobs

A cluster file system comprises a front-end file system, a back-end file system, data mover modules arranged between the front-end and back-end file systems, and a quota manager associated with at least a given one of data mover modules. The data mover modules are configured to control archiving of data between the front-end file system and the back-end file system for respective users based at least in part on respective user quotas established by the quota manager and identifying corresponding portions of the back-end file system available to the users. The front-end file system may comprise archive directories configured for respective ones of the users, with the data mover modules being configured to archive a given file from the front-end file system in the back-end file system responsive to a link to that file being stored in a corresponding one of the archive directories of the front-end file system.

Cluster file system comprising data mover modules having associated quota manager for managing back-end user quotas

A storage system comprises a shared storage environment that includes a storage array having at least one storage volume shared between first and second host devices. The storage system further comprises a server associated with the storage array, at least first and second clients associated with the respective first and second host devices, and a first block cache arranged between the first client and the storage array. The server is configured to coordinate operations of the first and second clients relating to the storage volume shared between the first and second host devices in a manner that ensures coherency of data stored in the first block cache. The server may comprise a storage block mapping protocol (SBMP) server and the first and second clients may comprise respective SBMP clients. The block cache is illustratively implemented using a VFCache or other type of server flash cache.

Block caching in a shared storage environment

Example embodiments of the present invention relate to a method and an apparatus for automatic cleanup of disfavored content. The method includes scanning a storage infrastructure for content having an attribute, identifying a disfavored instance of the content having the attribute, and removing the disfavored instance of the content from circulation.

Method and apparatus for automatic cleanup of disfavored content

Techniques are provided for scheduling computational tasks among multiple classes of storage resources based on a job classification. A job to be executed is classified into one of a plurality of predefined job classes. Each predefined job class is associated with a corresponding one of a plurality of predefined storage classes. The job is then assigned based on the classification to one of the storage resources of the predefined storage class associated with the classified predefined job class. Exemplary predefined storage classes include a performance class, a capacity class, a key-value storage class, and a shingled disk drive class. Exemplary predefined job classes include a CPU Intensive job class, an IO Intensive job class and a Small IO job class. Data required for a job is optionally prefetched before the job is assigned to a storage device. Data objects to be evicted from a storage device are optionally selected based on an anticipated future access.

Growing community of inventors

West Boylston, MA, United States of America

James M Pedone, Jr