Microsoft Technology Licensing, LLC

Ryota Tomioka

Matthew Alastair Johnson

Sebastian Nowozin

Alexander Lloyd Gaunt

Samuel Alexander Webster

Daniel Stefan Tarlow

Dimitrios Vytiniotis

Maik Riechert

Diane Bouchacourt

Frank Torsten Bernd Seide

Nuno Claudino Pereira Lopes

Bruno S Bozza

Renato Golin

Wilhelm Richert

Alberto Magni

Juliana Patrícia Vicente Franco

Siddharth Krishna

A neural network training apparatus is described which has a network of worker nodes each having a memory storing a subgraph of a neural network to be trained. The apparatus has a control node connected to the network of worker nodes. The control node is configured to send training data instances into the network to trigger parallelized message passing operations which implement a training algorithm which trains the neural network. At least some of the message passing operations asynchronously update parameters of individual subgraphs of the neural network at the individual worker nodes.

Asynchronous neural network training

A computation graph of a machine learning model is accessed from memory and a constraint solver is used to compute a partition of the computation graph into ordered stages of an execution pipeline. In use, when inference or training of the machine learning model takes place by executing the pipeline, execution cost of the stages are balanced according to the computed partition.

Partitioning for an execution pipeline

Disclosed in some examples are methods, systems, machine-readable media, and devices which operate a neural network defined by user code. A method includes identifying, operations from user code that are integral in operating the neural network, combining a subset of the identified operations into a single processing sequence to be transmitted to an array of hardware processors, performing operations that are not integral in operation of the neural network in a separate thread of execution from the operations that are integral in operating the neural network; and mapping results to the combined operations that were included in the single processing sequence.

Operating a neural network defined by user code

A data compression apparatus is described which has an encoder configured to receive an input data item and to compress the data item into an encoding comprising a plurality of numerical values. The numerical values are grouped at least according to whether they relate to content of the input data item or style of the input data item. The encoder has been trained using a plurality of groups of training data items grouped according to the content and where training data items within individual ones of the groups vary with respect to the style. The encoder has been trained using a training objective which takes into account the groups.

Data compression system

Growing community of inventors

Cambridge, United Kingdom

Ryota Tomioka