Sun Microsystems, Inc.

Oracle America, Inc.

Sandisk Enterprise IP LLC

Michael John Koster

Brian Walter O'Krafka

Darpan Dinker

Andrew David Eckhardt

Earl T Cohen

Thomas M McWilliams

Christopher Lyle Johnson

Ulrich Bruening

Thomas A Riddle

Shailendra Deva

Richard Jordan

Approaches for a non-volatile, solid-state storage system that is capable of supporting high bandwidth and/or random read/write access. The storage system may include a chassis having a bus slot and a disk bay, a master card mounted in the bus slot, and a flash memory card stacked in the disk bay and cabled to the master card. The master card enables one or more flash memory cards to be communicatively coupled to a single PCI Express bus. The master card may split a multi-lane PCI Express bus into a plurality of lanes, where one or more of the flash memory cards communicate over each of the plurality of lanes. Alternately, the master card may includes active circuitry for processing, switching, routing, reformatting, and/or converting the PCI Express bus into one or more busses for a plurality of flash memory cards. The stacked flash memory card is not in an enclosure.

Non-volatile solid-state storage system supporting high bandwidth and random access

Approaches for a distributed storage system that comprises a plurality of nodes. Each node, of the plurality of nodes, executes one or more application processes which are capable of accessing persistent shared memory. The persistent shared memory is implemented by solid state devices physically maintained on each of the plurality of nodes. Each the one or more application processes, maintained on a particular node, of the plurality of nodes, communicates with a shared data fabric (SDF) to access the persistent shared memory. The persistent shared memory comprises a scoreboard implemented in shared DRAM memory that is mapped to a persistent storage. The scoreboard provides a crash tolerant mechanism for enabling application processes to communicate with the shared data fabric (SDF).

Cluster of processing nodes with distributed global flash memory using commodity server technology

Data replication in a distributed node system including one or more nodes. A consensus protocol for failure recovery is implemented. Data items and information relating to consensus protocol roles of participant nodes are stored in at least some of the plurality of nodes. Logical logs stored in at least some of the plurality of nodes are created. The logical logs contain additional consensus protocol information including container metadata and replicated data.

Failure recovery using consensus replication in a distributed flash memory system

A Sharing Data Fabric (SDF) causes flash memory attached to multiple compute nodes to appear to be a single large memory space that is global yet shared by many applications running on the many compute nodes. Flash objects stored in flash memory of a home node are copied to an object cache in DRAM at an action node by SDF threads executing on the nodes. The home node has a flash object map locating flash objects in the home node's flash memory, and a global cache directory that locates copies of the object in other sharing nodes. Application programs use an applications-programming interface (API) into the SDF to transparently get and put objects without regard to the object's location on any of the many compute nodes. SDF threads and tables control coherency of objects in flash and DRAM.

Sharing data fabric for coherent-distributed caching of multi-node shared-distributed flash memory

A method and system for increasing programmability and scalability of a multi-processor network. A system includes two or more nodes coupled via a network with each node comprising a processor unit and memory. The processor unit includes one or more processors and a wiretap unit. The wiretap unit is configured to monitor memory accesses of the processors. A transaction may execute a number of read and/or write operations to memory. The nodes are configured to replicate one or more portions of memory; detect data conflicts to memory; and restore memory to pre-transaction state if needed.

Efficient hardware scheme to support cross-cluster transactional memory

A shared cache is point-to-point connected to a plurality of point-to-point connected processing nodes, wherein the processing nodes may be integrated circuits or multiprocessing systems. In response to a local cache miss, a requesting processing node issues a broadcast for requested data which is observed by the shared cache. If the shared cache has a copy of the requested data, the shared cache forwards the copy of the requested data to the requesting processing node.

Shared cache for point-to-point connected processing nodes

A method and system for increasing reliability and availability of a multi-processor network. A system includes a network with at least two nodes, with each node comprising a multi-processor unit (mpu) and memory. The mpu includes one or more processors and a wiretap unit. The wiretap unit and the memory included in the node are coupled to the processors in the node. The wiretap unit is configured to monitor memory accesses of the processors and convey data indicative of such accesses to a second node. The second node maintains a replica of memory in the first node, and is configured to undo modifications to the memory if needed. In the event of a hardware or software fault, the nodes are configured to restart the application on another node.

Maintaining memory checkpoints across a cluster of computing nodes

A multiprocessing node has a plurality of point-to-point connected microprocessors. Each of the microprocessors is also point-to-point connected to a filter. In response to a local cache miss, a microprocessor issues a broadcast for the requested data to the filter. The filter, using memory that stores a copy of the tags of data stored in the local cache memories of each of the microprocessors, relays the broadcast to those/microprocessors having copies of the requested data. If the snoop filter memory indicates that none of the microprocessors have a copy of the requested data, the snoop filter may either (i) cancel the broadcast and issue a message back to the requesting microprocessor, or (ii) relay the broadcast to a connected multiprocessing node.

Snooping-based cache-coherence filter for a point-to-point connected multiprocessing node

A proximity interconnect module includes a plurality of processors operatively connected to a plurality of off-chip cache memories by proximity communication. Due to the high bandwidth capability of proximity interconnect, enhancements to the cache protocol to improve latency may be made despite resulting increased bandwidth consumption.

Cache protocol enhancements in a proximity communication-based off-chip cache memory architecture

A proximity interconnect module includes a plurality of off-chip cache memories. Either disposed external to the proximity interconnect module or on the proximity interconnect module are a plurality of processors that are dependent on the plurality of off-chip cache memories for servicing requests for data. The plurality of off-chip cache memories are operatively connected to either one another or to one or more of the plurality of processors by proximity communication. Each of the plurality of off-chip cache memories may cache certain portions of the physical address space.

Proximity communication-based off-chip cache memory architectures

A proximity interconnect module includes a plurality of processors operatively connected to a plurality of off-chip cache memories by proximity communication. Due to the high bandwidth capability of proximity interconnect, when an off-chip cache memory is searched for requested data, either the requested data is at the same time searched for in on-chip cache memories of the proximity interconnect module or the requested data is at the same time retrieved from main memory. This reduces latency by reducing serial operations.

Speculative memory accesses in a proximity communication-based off-chip cache memory architecture

A cluster of multiprocessing nodes uses snooping-based cache-coherence to maintain consistency among the cache memories of the multiprocessing nodes. One or more of the multiprocessing nodes each maintain a directory table that includes a list of addresses of data last transferred by cache-to-cache transfer transactions. Thus, upon a local cache miss for requested data, a multiprocessing node searches its directory table for an address of the requested data, and if the address is found in the directory table, the multiprocessing node obtains a copy of the requested data from the last destination of the requested data as indicated in the directory table. Thereafter, a message indicating the completion of a cache-to-cache transfer is broadcast to other connected multiprocessing nodes on a 'best efforts' basis in which messages are relayed from multiprocessing node to multiprocessing node using low priority status and/or otherwise unused cycles.

Bandwidth reduction technique in a snooping-based cache-coherent cluster of multiprocessing nodes

A point-to-point connected multiprocessing node uses a snooping-based cache-coherence filter to selectively direct relays of data request broadcasts. The filter includes shadow cache lines that are maintained to hold copies of the local cache lines of integrated circuits connected to the filter. The shadow cache lines are provided with additional entries so that if newly referenced data is added to a particular local cache line by 'silently' removing an entry in the local cache line, the newly referenced data may be added to the shadow cache line without forcing the 'blind' removal of an entry in the shadow cache line.

Conservative shadow cache support in a point-to-point connected multiprocessing node

A multiprocessing node in a snooping-based cache-coherent cluster of processing nodes maintains a cache-to-cache transfer prediction directory of addresses of data last transferred by cache-to-cache transfers. In response to a local cache miss, the multiprocessing node may use the cache-to-cache transfer prediction directory to predict a cache-to-cache transfer and issue a restricted broadcast for requested data that allows only cache memories in the cluster to return copies of the requested data to the requesting multiprocessing node, thereby reducing the consumption of bandwidth that would otherwise be consumed by having a home memory return a copy of the requested data in response to an unrestricted broadcast for requested data that allows cache memories and home memories in a cluster to return copies of the requested data to the requesting multiprocessing node.

Growing community of inventors

Bridgeville, CA, United States of America

Michael John Koster