Hewlett Packard Enterprise Development LP

Silicon Graphics, Incorporated

Michael S Woodacre

Thomas Edward McGee

Michael E Malewicki

Dejan S Milojicic

Daniel E Lenoski

Andrew Ray Wheeler

Frank R Dropps

William A Huffman

Randal Steven Passint

Jeffrey Scott Kuskin

Derek Steven Schumacher

Kirill Malkin

Keith Packard

Randy Passint

Sharad Singhal

Steven C Miller

Steven J Dean

Eric Carl Fromm

Eng Lim Goh

Russell Leonard Nicol

Steve Dean

George Hopkins

Paul Frank

Stuart C Haden

Brian J Johnson

Kevin Knecht

One aspect of the application can provide a system and method for replacing a failing node with a spare node in a non-uniform memory access (NUMA) system. During operation, in response to determining that a node-migration condition is met, the system can initialize a node controller of the spare node such that accesses to a memory local to the spare node are to be processed by the node controller, quiesce the failing node and the spare node to allow state information of processors on the failing node to be migrated to processors on the spare node, and subsequent to unquiescing the failing node and the spare node, migrate data from the failing node to the spare node while maintaining cache coherence in the NUMA system and while the NUMA system remains in operation, thereby facilitating continuous execution of processes previously executed on the failing node.

Chassis servicing and migration in a scale-up NUMA system

Example implementations relate to cache coherency protocols as applied to a memory block range. Exclusive ownership of a range of blocks of memory in a default shared state may be tracked by a directory. The directory may be associated with a first processor of a set of processors. When a request is received from a second processor of the set of processors to read one or more blocks of memory absent from the directory, one or more blocks may be transmitted in the default shared state to the second processor. The blocks absent from the directory may not be tracked in the directory.

Application of a default shared state cache coherency protocol

One embodiment can provide a node controller in a multiprocessor system. The node controller can include a processor interface to interface with a processor, a memory interface to interface with a fabric-attached memory, a node-controller interface to interface with a remote node controller, and a cache-coherence logic to operate in a first mode or a second mode. The cache-coherence logic manages cache coherence for a local memory of the processor coupled to the processor interface when operating in the first mode, and the cache-coherence logic manages cache coherence for the fabric-attached memory coupled to the memory interface when operating in the second mode.

System and method for scalable hardware-coherent memory nodes

A node controller is provided to include a first interface to interface with one or more processors, a second interface including a plurality of ports to interface with node controllers within a base node and other nodes in the cache-coherent interconnect network. The node controller can further include a third interface to interface with a first plurality of memory devices and a cache coherence management logic. The cache coherence management logic can maintain, based on a first circuitry, hardware-managed cache coherency in the cache-coherent interconnect network. The cache coherence management logic can further facilitate, based on a second circuitry, software-managed cache coherency in the cache-coherent interconnect network.

System and method for facilitating hybrid hardware-managed and software-managed cache coherency for distributed computing

In exemplary aspects of cache coherency management, a first request is received and includes an address of a first memory block in a shared memory. The shared memory includes memory blocks of memory devices associated with respective processors. Each of the memory blocks are associated with one of a plurality of memory categories indicating a protocol for managing cache coherency for the respective memory block. A memory category associated with the first memory block is determined and a response to the first request is based on the memory category of the first memory block. The first memory block and a second memory block are included in one of the same memory devices, and the memory category of the first memory block is different than the memory category of the second memory block.

Cache coherency management for multi-category memories

Systems and methods for dynamically and programmatically controlling hardware and software to optimize bandwidth and latency across partitions in a computing system are discussed herein. In various embodiments, performance within a partitioned computing system may be monitored and used to automatically reconfigure the computing system to optimize aggregate bandwidth and latency. Reconfiguring the computing system may comprise reallocating hardware resources among partitions, programming firewalls to enable higher bandwidth for specific inter-partition traffic, switching programming models associated with individual partitions, starting additional instances of one or more applications running on the partitions, and/or one or more other operations to optimize the overall aggregate bandwidth and latency of the system.

Systems and methods for aggregate bandwidth and latency optimization

In high performance computing, the potential compute power in a data center will scale to and beyond a billion-billion calculations per second ('Exascale' computing levels). Limitations caused by hierarchical memory architectures where data is temporarily stored in slower or less available memories will increasingly limit high performance computing systems from approaching their maximum potential processing capabilities. Furthermore, time spent and power consumed copying data into and out of a slower tier memory will increase costs associated with high performance computing at an accelerating rate. New technologies, such as the novel Zero Copy Architecture disclosed herein, where each compute node writes locally for performance, yet can quickly access data globally with low latency will be required. The result is the ability to perform burst buffer operations and in situ analytics, visualization and computational steering without the need for a data copy or movement.

Method and system for shared direct access storage

An apparatus, system, and method of enforcing cache coherency in a multiprocessor shared memory system are disclosed. A request is received from a node controller, to process a cache coherent operation on a memory block in a shared memory. Based on the information included in the request, a determination is made as to whether the request was transmitted from a processor that is remote relative to the memory that includes the memory block referenced in the request. If the request is from a remote processor, a hardware-based cache coherency of the system is disabled, and request is processed according to software-based cache coherency protocols and mechanisms. A coherent read request may be translated to a non-coherent request, such as an immediate read request, which does not trigger tracking or storing state and ownership information of the requested memory block, or trigger communications with processors other than those involved with request. Processing a coherent write request may include transmitting an exclusive read request, which is a request for ownership of the memory block identified in the coherent write request, and transmitting a write acknowledgment to the node controller.

Selective disabling of hardware-based cache coherency and enforcement of software-based cache coherency

Example implementations relate to an apparatus to support providing a computing service to a client including transferring control between a primary data processing system and a secondary data processing system in response to an event; the primary data processing system comprising a processor and associated memory and the secondary data processing system comprising a processor and associated memory; the apparatus comprising: circuitry to identify restoration data; the restoration data comprising at least data associated with at least one predetermined type of memory operation of the memory associated with the primary data processing system, and circuitry to output any identified restoration data for storage in the memory associated with the processor of the secondary data processing system.

Data processing apparatuses and methods to support transferring control between a primary data processing system and a secondary data processing system in response to an event

Systems and methods for dynamically modifying coherence domains are discussed herein. In various embodiments, a hardware controller may be provided that is configured to automatically recognize application behavior and dynamically reconfigure coherence domains in hardware and software to tradeoff performance for reliability and scalability. Modifying the coherence domains may comprise repartitioning the system based on cache coherence independently of one or more software layers of the system. Memory-driven algorithms may be invoked to determine one or more dynamic coherence domain operations to implement. In some embodiments, declarative policy statements may be received from a user via one or more interfaces associated with the controller. The controller may be configured to dynamically adjust cache coherence policy based on the declarative policy statements received from the user.

Systems and methods for dynamically modifying coherence domains

A high performance computing (HPC) system has an architecture that separates data paths used by compute nodes exchanging computational data from the data paths used by compute nodes to obtain computational work units and save completed computations. The system enables an improved method of saving checkpoint data, and an improved method of using an analysis of the saved data to assign particular computational work units to particular compute nodes. The system includes a compute fabric and compute nodes that cooperatively perform a computation by mutual communication using the compute fabric. The system also includes a local data fabric that is coupled to the compute nodes, a memory, and a data node. The data node is configured to retrieve data for the computation from an external bulk data storage, and to store its work units in the memory for access by the compute nodes.

Workload management system and process

An apparatus and method exchange data between two nodes of a high performance computing (HPC) system using a data communication link. The apparatus has one or more processing cores, RDMA engines, cache coherence engines, and multiplexers. The multiplexers may be programmed by a user application, for example through an API, to selectively couple either the RDMA engines, cache coherence engines, or a mix of these to the data communication link. Bulk data transfer to the nodes of the HPC system may be performed using paged RDMA during initialization. Then, during computation proper, random access to remote data may be performed using a coherence protocol (e.g. MESI) that operates on much smaller cache lines.

Caching network fabric for high performance computing

A method and system for managing memory in a multiprocessor system includes defining the plurality of processor coherence domains within a system coherence domain of the multiprocessor system. The processor coherence domains each include a plurality of processors and a processor memory. Shared access to data in the processor memory of each processor coherence domain is provided only to elements of the multiprocessor system within the processor coherence domain. Non-shared access to data in the processor memory of each processor coherence domain is provided to elements of the multiprocessor system within and outside of the processor coherence domain.

Growing community of inventors

Winchester, United Kingdom

Michael S Woodacre