Graphcore Limited

Oracle America, Inc.

Oracle International Corporation

Sun Microsystems, Inc.

Ola Torudbakken

Bjorn Dag Johnsen

Brian Manula

Andreas V Bechtolsheim

Harald Hoeg

Lorenzo Cevolani

Simon Christian Knowles

Daniel John Pelham Wilkinson

Rabin Sugumar

Lars Paul Huse

Steinar Forsmo

Inge Lars Birkeli

Ashok V Krishnamoorthy

Xuezhe Zheng

Arvind Srinivasan

Stephen Felix

Martin Paul Mayhead

Hans Rygh

Naresh Revanuru

Adam Messinger

Oleksandr Otenko

Milton C Lee

Ballav Bihani

David Brean

Morten Schanke

Steven Liu

Haakon Ording Bugge

Daniel Bernard Hruska

Predrag Hodoba

Magne Vigulf Sandven

Nathan Luther Reynolds

Bjorn-Dag Johnsen

Patrik Torstensson

Joshua Lee Dorr

Gilberto Figueroa

Lenny Phan

Staffan Larsen

Pallab Bhattacharya

Scott Oaks

Roy Arntsen

Patrick J Decker

Fabian Tschopp

Brad Cameron

Hon Hung Yam

Wei-Lin Guay

A gateway for interfacing a host with a subsystem for acting as a work accelerator to the host. The gateway enables the transfer of batches of data to the subsystem at precompiled data exchange synchronisation points. The gateway acts to route data between accelerators which are connected in a scaled system of multiple gateways and accelerators using a global address space set up at compile time of an application to run on the computer system.

Gateway fabric ports

A host system compiles a set of local programs which are provided over a network to a plurality of subsystems. By defining the synchronisation activity on the host, and then providing that information to the subsystems, the host can service a large number of subsystems. The defined synchronisation activity includes defining the synchronisation groups between which synchronisation barriers occur and the points during program execution at which data exchange with the host occurs. Defining synchronisation activity between the subsystems allows a large number of subsystems to be connecting whilst minimising the required exchanges with the host.

Barrier synchronization between host and accelerator over network

A data processing system comprising a plurality of processing nodes that are arranged to update a model in a parallel manner Each of the processing nodes starts with a different set of updates to model parameters. Each of the processing nodes is configured to perform one or more reduce-scatter collectives so as to exchange and reduce the updates. Having done so, each processing node is configured to apply the reduced set of updates to obtain an updated set of model parameters. The processing nodes then exchange the updated model parameters using an all-gather so that each processing node ends up with the same model parameters at the end of the process.

Update of model parameters in a parallel processing system

A data processing system comprising a plurality of processing nodes, each comprising at least one memory configured to store an array of data items, wherein each of the plurality of processing nodes is configured to execute compute instructions during a compute phase and following a precompiled synchronisation barrier, enter at least one exchange phase. During the at least one exchange phase, a series of collective operations are carried out. Each processing node is configured to perform a reduce scatter collective in at least one first dimension. Using the results of the reduce scatter collective, each processing node performs an allreduce in a second dimension. The processing nodes then perform an all-gather collective in the at least one first dimension using the results of the allreduce.

Control of data transfer between processing nodes

In a data processing system, at least one processing node is configured to perform computations for a multi-stage process whilst at least one other processor performs the load/unload operations required to calculate a subsequent stage of the multi stage process. An exchange of data then occurs between the processing nodes. At a later time, at least one processing node performs calculations using the data loaded from storage, whilst at least one other processor performs the load/unload operations required to calculate a subsequent stage of the multi stage process.

Control of processing node operations

A gateway for use in a computing system to interface a host with the subsystem for acting as a work accelerator to the host, the gateway having an streaming engine for controlling the streaming of batches of data into and out of the gateway in response to pre-compiled data exchange synchronisation points attained by the subsystem, wherein the streaming of batches of data is selectively via at least one of an accelerator interface, a data connection interface, a gateway interface and an memory interface, wherein the streaming engine is configured to perform data preparation processing of the batches of data streamed into the gateway prior to said batches of data being streamed out of the gateway, wherein the data preparation processing comprises at least one of: data augmentation; decompression; and decryption.

Gateway processing

According to an aspect of the invention, there is provided a computer comprising a plurality of interconnected processing nodes arranged in a configuration with multiple stacked layers. Each layer comprises four processing nodes connected by respective links between the processing nodes. In end layers of the stack, the four processing nodes are interconnected in a ring formation by two links between the nodes, the two links adapted to operate simultaneously. Processing nodes in the multiple stacked layers provide four faces, each face comprising multiple layers, each layer comprising a pair of processing nodes. The processing nodes are programmed to operate a configuration to transmit data around embedded one-dimensional rings, each ring formed by processing nodes in two opposing faces.

Networked computer with multiple embedded rings

A system comprising a gateway for interfacing external data sources with one or more accelerators. The gateway comprises a plurality of virtual gateways, each of which is configured to stream data from the external data sources to one or more associated accelerators. The plurality of virtual gateways are each configured to stream data from external data sources so that the data is received at an associated accelerator in response to a synchronisation point being obtained by a synchronisation zone. Each of the virtual gateways is assigned a virtual ID so that when data is received at the gateway, data can be delivered to the appropriate gateway.

Virtualised gateways

A computer system comprises a work accelerator, a gateway the transfer of data to the accelerator from external storage, the accelerator executes a first compiled code sequence to perform computations on data transferred to the accelerator from the gateway. The first compiled code sequence comprises a synchronisation instruction indicating a barrier between a compute phase in which the compute instructions are executed and an exchange phase, wherein execution of the synchronisation instruction causes an indication of a pre-compiled data exchange synchronisation point to be transferred to the gateway. The gateway comprises a streaming engine storing a second compiled code sequence in the form of a set of data transfer instructions executable by the streaming engine to perform data transfer operations to stream data through the gateway in the exchange phase, wherein the first and second compiled code sequences are generated as a related set at compile time.

Code compilation for scaling accelerators

One aspect of the invention provides a computer comprising a plurality of interconnected processing nodes arranged in a ladder configuration comprising a plurality of facing pairs of processing nodes. The processing nodes of each pair are connected to each other by two links. A processing node in each pair is connected to a corresponding processing node in an adjacent pair by at least one link. The processing nodes are programmed to operate the ladder configuration to transmit data around two embedded one-dimensional rings formed by respective sets of processing nodes and links, each ring using all processing nodes in the ladder once only.

Networked computer with embedded rings field

System and method for supporting flexible partition key (P_Key) mapping in a high performance computing environment. In accordance with an embodiment, when such computer environments comprise more than one subnet and/or forwarding domain that support inter-subnet partitions, the P_Keys comprising such inter-subnet partitions can change across domain boundaries. In such situations, a partition key of a packet can be updated upon entry to a new subnet and/or forwarding domain.

System and method for supporting flexible P_Key mapping in a high performance computing environment

When an unsafe port with a loss of signal is detected, a transceiver may enable one laser in a group of lasers associated with the unsafe port and may disable the remaining lasers. Then, the transceiver may instruct a transmitter associated with the one laser to transmit an optical signal on the unsafe port using a reduced transmit power that is less than a threshold value associated with the Class 1 conditions and at a different time than enabled lasers in other groups of lasers. Alternatively, for a safe port on which valid communication is received, the transceiver may enable lasers in a group of lasers associated with the safe port. Then, the transceiver may instruct transmitters associated with the lasers in this group of lasers to transmit optical signals on the safe port using a normal transmit power for the lasers that is greater than the threshold value.

Safety-enhanced laser array

Growing community of inventors

Oslo, Norway

Ola Torudbakken