Microsoft Technology Licensing, LLC

Dimitrios Vytiniotis

Kapil Vaswani

Matthew Alastair Johnson

Manuel Silverio Da Silva Costa

Samuel Alexander Webster

Alexander Lloyd Gaunt

Matthew John Parkinson

Daniel Stefan Tarlow

Ryota Tomioka

Maik Riechert

Antonio Criminisi

Christos Gkantsidis

Antony Rowstron

Dushyanth Narayanan

Orion Tamlin Hodson

Aditya Vithal Nori

Osbert Bastani

Leonidas Lampropoulos

Piyus Kumar Kedia

A neural network training apparatus is described which has a network of worker nodes each having a memory storing a subgraph of a neural network to be trained. The apparatus has a control node connected to the network of worker nodes. The control node is configured to send training data instances into the network to trigger parallelized message passing operations which implement a training algorithm which trains the neural network. At least some of the message passing operations asynchronously update parameters of individual subgraphs of the neural network at the individual worker nodes.

Asynchronous neural network training

A method of manual memory management is described which comprises enabling one or more threads to access an object created in a manual heap by storing a reference to the object in thread-local state and subsequently deleting the stored reference after accessing the object. In response to abandonment of the object, an identifier for the object and a current value of either a local counter of a thread or a global counter are stored in a delete queue and all threads are prevented from storing any further references to the object in thread-local state. Deallocation of the object only occurs when all references to the object stored in thread-local state for any threads have been deleted and a current value of the local counter for the thread or the global counter has incremented to a value that is at least a pre-defined amount more than the stored value, wherein the global counter is updated using one or more local counters.

Safe manual memory management

A method of manual memory management is described. In response to detecting an access violation triggered by the use of an invalid reference to an object in a manual heap, a source of the access in a register or stack is identified. An updated reference for the object using stored mapping data is determined and used to replace the invalid reference in the source.

Manual memory management using lazy patching

A training engine is described which has a memory arranged to access a neural network image classifier, the neural network image classifier having been trained using a plurality of training images from an input space, the training images being labeled for a plurality of classes. The training engine has an adversarial example generator which computes a plurality of adversarial images by, for each adversarial image, searching a region in the input space around one of the training images, the region being one in which the neural network is linear, to find an image which is incorrectly classified into the plurality of classes by the neural network. The training engine has a processor which further trains the neural network image classifier using at least the adversarial images.

Neural network image classifier

Methods of generating filters automatically from data processing jobs are described. In an embodiment, these filters are automatically generated from a compiled version of the data processing job using static analysis which is applied to a high-level representation of the job. The executable filter is arranged to suppress rows and/or columns within the data to which the job is applied and which do not affect the output of the job. The filters are generated by a filter generator and then stored and applied dynamically at a filtering proxy that may be co-located with the storage node that holds the data. In another embodiment, the filtered data may be cached close to a compute node which runs the job and data may be provided to the compute node from the local cache rather than from the filtering proxy.

Generation and application of correctness-enforced executable filters

Growing community of inventors

Cambridge, United Kingdom

Dimitrios Vytiniotis