International Business Machines Corporation

Google Inc.

Rajagopal Ananthanarayanan

Robert Michael Rees

Renu Tewari

Manpreet Singh

Ralph Attila Becker-Szendy

Ashish Gupta

Haifeng Jiang

Tianhao Qiu

Sumit Das

Venkatesh Basker

Dharmendra S Modha

Frank Bernhard Schmuck

Dean Hildebrand

Manoj Premanand Naik

Venkateswararao Jujjuri

Roger Lee Haskin

Craig Fulmer Everhart

Marc M Eshel

Jason Christopher Young

Randal Chilton Burns

Darrell D E Long

David Macnaughton Wolfe

Karan Gupta

Prasenjit Sarkar

Steven Kyle Esser

Prashant Pandey

Raghavendra Singh

Himabindu Pucha

D Scott Guthridge

James J Seeger

Mansi Ajit Shah

Jujjuri Venkateswararao

Shyamal S Chandra

A system facilitates access to data in a network and includes a cache that stores instructions. A processor executes the instructions including: caching processing configured to integrate caching into a local cluster file system, and cache local file data in the cache based on fetching file data on demand from a remote cluster file system. The cache is visible to file system clients as a Portable Operating System Interface (POSIX) compliant file system. Applications execute on a multi-node cache cluster using POSIX semantics via a POSIX compliant file system interface. Data cache is locally and remotely consistent for updates.

Scalable caching of remote file data in a cluster file system

A system facilitates access to data in a network. One implementations includes a caching layer function that is configured to integrate into a local cluster file system, to cache local file data in a cache based on fetching file data on demand from a remote cluster file system, and to operate on a multi-node cache cluster. The cache is visible to file system clients as a Portable Operating System Interface (POSIX) compliant file system, applications execute on the multi-node cache cluster using POSIX semantics via a POSIX compliant file system interface, and data cache is locally consistent for updates made at the multi-node cache cluster using distributed locking for the data cache.

Aspects of the disclosure relate generally to reducing latency for consensus in geographically distributed disaster-safe persistent data-store systems. These distributed systems may include registry system having redundant storage for maintaining the system status. Each registry system may include a server and a storage component. Consensus may be achieved by querying all of other servers of the registry system. In one example, the consensus data may be sharded into independent small groups. This may allow for multiple consensus transactions to be generated and run in parallel, which, in turn may reduce the latency. In addition, or alternatively, requests to a server to write or otherwise change the data-store may be batched at the server side. Thus, for the consensus, the server need only communicate with the other servers only once per batch. This may also reduce the latency of the distributed system.

Latency reduction in distributed computing systems

Aspects of the disclosure relate generally to a distributed event joining system for joining primary and secondary events based on a common key. In one example, a continuous stream of primary events may be submitted by a plurality of ad serving computers for storage at a cache layer. Each primary event may be associated with a primary event key. After a time T, the stream of primary events may be sent to long term memory. The system may also receive secondary events, each associated with a primary key. The primary key may be used to identify a primary event in the cache layer. If the primary event is not available in the cache layer, the long term memory may be searched for the primary key. Paired events may then be submitted to a distributed joined event storage system.

Multi-level caching event lookup

Aspects of the disclosure relate generally to a distributed event joining system for joining primary and secondary events based on a common key. In one example, a continuous stream of primary events may be stored at a cache layer. Each primary event may be associated with a primary event key which may be used to generate an index map for the cache layer. After a time T, the stream of primary events may be sent to long term memory. The system may also receive secondary events, each associated with a primary key. The primary key may be used to identify a primary event in the cache layer. If the primary event is not available in the cache layer, the long term memory may be searched for the primary key. Paired events may then be submitted to a distributed joined event storage system.

Real-time log joining on a continuous stream of events that are approximately ordered

Embodiments of the invention relate to optimizing a file system for different types of applications in a compute cluster using dynamic block size granularity. An exemplary embodiment includes reserving a predetermined number of storage allocation regions for each node in a cluster, wherein each storage allocation region comprises a set of contiguous data blocks on a single storage disk of locally attached storage; using a contiguous set of data blocks on a single storage disk of locally attached storage as a file system's block allocation granularity for striping data to be stored in the file system for a compute operation in the cluster; and using a single data block of a shared storage subsystem as the file system's block allocation granularity for striping data to be stored in the file system for a data access operation in the cluster.

Optimizing a file system for different types of applications in a compute cluster using dynamic block size granularity

Adaptive and integrated visualization of spatiotemporal data from large-scale simulation, is provided. A simulation is performed utilizing a simulator comprising multiple processors, generating spatiotemporal data samples from the simulation. Each data sample has spatial coordinates with a time stamp at a specific time resolution, and a tag. The data samples are assembled into data streams based on at least one of a spatial relationship and the corresponding tag. Each data stream is encoded into multiple formats, and an integrated and adaptive visualization of the data streams is displayed, wherein various data streams are simultaneously and synchronously displayed.

Adaptive and integrated visualization of spatiotemporal data from large-scale simulations

Access to objects of a computing environment is managed. Access to an object of the computing environment is prohibited via one cached entity corresponding to the object, while access to the object is allowed via another cached entity. The cached entities are concurrently maintained for a period of time, but the one cached entity is in a shadow state. In one example, the cached entities are session locks and the object is a file system object.

Managing access to objects of a computing environment

Neuronal networks of electronic neurons interconnected via electronic synapses with synaptic weight normalization. The synaptic weights are based on learning rules for the neuronal network, such that a synaptic weight for a synapse determines the effect of a spiking source neuron on a target neuron connected via the synapse. Each synaptic weight is maintained within a predetermined range by performing synaptic weight normalization for neural network stability.

Synaptic weight normalized spiking neuronal networks

A system and method for managing logical versions of a filesystem made through a near-instantaneous copy process is provided. Each logical version of the filesystem, including the primary version, is assigned a unique epoch number or other identifying value, and each filesystem object is assigned two epoch numbers that define the space of logical versions for which the object exists. A list of all valid epoch numbers are maintained in a version table. The near-instantaneous creation of a logical version of the filesystem requires inserting a unique epoch number entry in the version table () and the next epoch number to the new logical version. An original state of the system is preserved through the use of a copy-on-write procedure for filesystem objects that are reference by that logical filesystem version (). Accordingly, filesystem versions are maintained and original states of previous states of the logical filesystem versions are preserved.

Managing filesystem versions

A method and system for asserting a lock in a distributed file system is provided. All distributed locks have a lease for a limited time period, wherein the lease may be renewed periodically. A lock manager data structure is provided to maintain mode compatibility of locks granted to different client nodes for the same object. The process of acquiring or reasserting a lock includes determining whether there are other valid locks in use, as well as whether a valid lock is in operation in a conflicting or compatible mode with a requested lock. A new lock lease may be granted and a lock lease which has expired may be reasserted if a conflicting lease is not present.

Discipline for lock reassertion in a distributed file system

Various copy-on-write implementations for a computing environment are presented. One copy-on-write implementation includes employing a read mapping table to perform a first virtual block to physical block mapping for use in reading a block of data of a file to be copy-on-written from physical storage for modification; and employing a different, write mapping table to perform a second virtual block to physical block mapping for use in writing a modified block of the file data to physical storage, wherein copy-on-write of the block of data is achieved using a single write operation. In another implementation, a distributed copy-on-write of a file for a client server environment is presented. This distributed copy-on-write includes performing, by a first client, copy-on-write of at least one block of data of the file to be copy-on-written, and performing, by a second client, copy-on-write of at least one other block of data of the file, wherein multiple clients perform the copy-on-write of the file.

Growing community of inventors

San Jose, CA, United States of America

Rajagopal Ananthanarayanan