International Business Machines Corporation

Michael Edward Factor

Gail Andrea Spear

Shachar Fienblit

Elliot Karl Kolodner

Aviad Zlotnick

William Frank Micka

Thomas Charles Jarvis

Robert Francis Bartfai

Olympia Gluck

Paula Kim Ta-Shma

Sam Clark Werner

Alain Charles Azagury

David Hadas

Amiram Hayardeny

Ifat Nuriel

Rivka Mayraz Matosevich

Danny Harnik

Shmuel Ben-Yehuda

Alexandra Shulman-Peleg

Dalit Tzafrir

Kenneth Wayne Boyd

Ronen Itshak Kat

Guy Laden

Sheli Rahav

Sivan Tal

Joseph S Hyde, Ii

Martin Tross

Warren Keith Stanley

Dmitry Sotnikov

Dalit Naor

Kalman Zvi Meth

Eli Malul

Shlomit Sarah Pinter

Benjamin Mandler

Avishay Traeger

Jason K Resch

Andrew D Baptist

Wesley B Leggette

Ethan S Wozniak

Joseph S Glider

Orit Nissan-Messing

John Jay Wolfgang

Alessandro Sorniotti

Kenneth F Day, Iii

Gal Ashour

Vita Bortnikov

Yariv Aridor

Anil Kurmus

Amir Epstein

Demyn Lee Plantenberg

Edward Hsiu-Wei Lin

Itzhack Goldberg

Gregory E McBride

Ravi V Khadiwala

Carol Santich Mellgren

William Joseph Armstrong

Neil Sondhi

Robert Akira Kubo

Robert Bruce Nicholson

William James Scales

Robert Haas

Robert Beverley Basham

Michael Hirsch

Eitan D Farchi

Matthew Albert Huras

Kenneth Fairclough Day, Iii

Tamar Eilam

Zorik Machulsky

Yoelle Smadja Maarek

Eran Raichstein

Robert Frederic Kern

Dan S Pelleg

Eyal Gordon

Roberta Jo Cochrane

Edward Lin

David Blair Whitworth

Harsha Hegde

Abel Alkon Gordon

William W Owen

Yoram Novick

Thomas D Weigold

Aamer U Sachedina

Ilan Shimony

David Breitgand

Slavisa Sarafijanovic

Assaf Schuster

Simon Lorenz

Lee Charles LaFrese

John Earle Lindley

Yosef Moatti

Gary Valentin

Dean Har'el Lorenz

Harald Seipp

Dafna Sheinwald

Cataldo Mega

Martin John Trotter

Sima Nadler

Ben-Ami Yassour

Naama Kraus

John Thomas Flynn, Jr

An example system includes a processor to generate a data plane based on dataset metadata, data plane component metadata, data governance decisions, and information technology (IT) system metrics. The processor is to also apply components of the data plane for a specified workload across a number of computing devices selected by a control plane.

Generating data planes for processing of data workloads

Embodiments of the present systems and methods may provide the capability ensure that data is persisted and accessed correctly without depending on eventually consistent list operations on the object store. For example, in an embodiment, a computer-implemented method for data distribution may comprise attempting to persist a plurality of data parts from a plurality of processing tasks, generating a manifest including information indicating those attempts to persist data parts that have succeeded, and persisting the manifest with the data parts that have been successfully persisted.

Persisting distributed data sets into eventually consistent storage systems

Embodiments of the present systems and methods may provide techniques to provide host side encryption while maintaining compression and deduplication benefits and providing communication between the host and the storage system that does not leak information about the data compressibility/deduplication properties. For example, in an embodiment, a method may comprise compressing, at a computer system, an original sector of data, generating a new sector of data including a first part including metadata and padding data, and a second part including the original sector of data that has been compressed and encrypted using a data encryption key (DEK), encrypting, at the computer system, the new sector of data using a data reduction key (DRK), and transmitting, at the computer system, the encrypted new sector of data to a storage system.

Key-compressible encryption

An example operation may include one or more of dividing a data object into a plurality of parts, hashing the data object to generate a hashed data object and hashing the plurality of parts to generate a plurality of hashed parts, generating a plurality of slices where each slice includes the hashed data object and a different respective hashed part, and distributing the plurality of slices across a plurality of blockchain peers, respectively.

Efficient threshold storage of data object

A storage system access point receives a first access request from a client device, wherein the first access request specifies first data. The storage system access point attempts to execute the first access request. In response to successfully accessing the first data, a first message is transmitted to an external audit system indicating that information corresponding to the first access request is to be recorded by the external audit system. In response to that the first data has been successfully accessed, and that the information corresponding to the first access request has been successfully recorded by the external audit system, notifying the client device that the first access request has been successfully completed.

Atomically tracking transactions for auditability and security

Embodiments of the present systems and methods may provide techniques to provide simple and accurate estimate of memory requirements for application invocation in a serverless environment. For example, a method may comprise selecting sample invocations of functions as a service from a larger plurality of invocations, submitting for execution the plurality of sample invocations and, for each sample invocation, submitting a specification of a memory size to be used for execution of each sample invocation, determining, whether the specification of the memory size to be used for execution of each sample invocation results in unsuccessful execution of at least some of the sample invocations due to insufficient memory and, if so, adjusting the specification of the memory size for at least some of the sample invocations, and submitting for execution at least those invocations in the larger plurality of invocations that were not included in the plurality of sample invocations.

Dynamic determination of memory requirements for function as a service multi-invocation flows

A method of integrating a distributed ledger represented by a blockchain with a distributed storage network (DSN) begins by sending a proof of existence request to the DSN, the proof of existence request including an object name, an object version, a start time and an end time. The method continues by reading the object metadata for the sent object name. The method continues by checking a revision history from object metadata associated with the sent object name to ensure the object existed by the start time through the end time with no deletions and, if the object did not exist by the start time through the end time with no deletions, rejecting the proof of existence request and returning an error response. If the object did exist by the start time on through the end time with no deletions, the method continues by producing and returning an attestation comprising verification information related to the object.

Producing proof of receipt, existence and other data provenance evidence

In some examples, a system for executing instructions can include a processor to detect data to be transmitted to a storage device in response to a write operation. The processor can also determine that the data comprises a compressible characteristic that enables compression of the data to a size below a threshold value. Additionally, the processor can generate a modified data block by encrypting the compressed data, and adding a padding to the compressed and encrypted data. Furthermore, the processor can transmit the modified data block to the storage device.

Encrypted and compressed data transmission with padding

A method begins by a processing module identifying, for a DSN (Dispersed Storage Network) memory using multiple IDA (Information Dispersal Algorithms) configurations simultaneously, a first IDA configuration with a highest security level relative to each of the multiple IDA configurations. The method continues by generating at least one master key. The method continues by encoding the master key with a secure error coding function to produce master key slices according to the first IDA configuration. The method continues by storing the master key slices in the DSN memory using the first IDA configuration. The method continues by, when storing data with a second IDA configuration having a security level lower than the first IDA configuration, retrieving the master key slices, decoding the master key slices to obtain the master key and encrypting the data using the master key.

Enhancing security for multiple storage configurations

An object-based data storage system includes a memory and a processor for executing machine executable instructions configured for implementing logical containers for data objects each having a global identifier. The containers are configured for storing metadata including a first parameterization value descriptive of a number of storage nodes and a second parameterization value descriptive of a classification of the data objects. The machine executable instructions are further configured for implementing a first object storage ring for addressing storage locations across the multiple storage nodes using a surjective function. Execution of the instructions causes the processor to: instantiate the first object storage ring, receive the global identifier and the metadata by the first object storage ring, and generate a storage address by the first object storage ring for the data object using the global identifier, the first parameterization value and the second parameterization value as input to the surjective function.

Data object storage across multiple storage nodes

Machines, systems and methods for recovering data objects in a distributed data storage system, the method comprising storing one or more replicas of a first data object on one or more clusters in one or more data centers connected over a data communications network; recording health information about said one or more replicas, wherein the health information comprises data about availability of a replica to participate in a restoration process; calculating a query-priority for the first data object; querying, based on the calculated query-priority, the health information for the one or more replicas to determine which of the one or more replicas is available for restoration of the object data; calculating a restoration-priority for the first data object based on the health information for the one or more replicas; and restoring the first data object from the one or more of the available replicas, based on the calculated restoration-priority.

Automated data recovery from remote data object replicas

Growing community of inventors

Haifa, Israel

Michael Edward Factor