Royal Bank of Canada

Ruitong Huang

Weiguang Ding

Yanshuai Cao

Yik Chau Lui

Luyu Wang

Chenjun Xiao

Junfeng Wen

Xiaomeng Jin

Kry Yik Chau Lui

Yash Sharma

A system, electronic device and method for improved neural network training are provided. The electronic device includes: a processor, a memory storing a Generative adversarial network (GAN) to learn from unlabeled data by engaging a generative model in an adversarial game with a discriminator; and one or more programs stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for training the Generative adversarial network using a regularizer to encourage the discriminator to properly use its capacity and hidden representations of the discriminator to have high entropy.

System and method for improved neural network training

An improved computer implemented method and corresponding systems and computer readable media for improving performance of a deep neural network are provided to mitigate effects related to catastrophic forgetting in neural network learning. In an embodiment, the method includes storing, in memory, logits of a set of samples from a previous set of tasks (D); and maintaining classification information from the previous set of tasks by utilizing the logits for matching during training on a new set of tasks (D).

System and method for improving deep neural network performance

Systems and methods of updating a multi-level data structure for controlling an agent. The method may include: accessing a data structure defining one or more nodes. A non-leaf node of the one or more nodes may be associated with one or more edges for traversing to a subsequent node. An edge of the one or more edges may be associated with a visit count and a softmax state-action value estimation. The method may include identifying a node trajectory including a series of nodes based on an asymptotically converging sampling policy, where the node trajectory includes a root node and a leaf node of the data structure, determining a reward indication associated with the node trajectory; and for at least one non-leaf node, updating the visit count and the softmax state-action value estimate associated with one or more edges of the non-leaf node based on the determined reward indication.

System and method for tree-based machine learning

A system for generating an adversarial example in respect of a neural network, the adversarial example generated to improve a margin defined as a distance from a data example to a neural network decision boundary. The system includes a data receiver configured to receive one or more data sets including at least one data set representing a benign training example (x); an adversarial generator engine configured to: generate, using the neural network, a first adversarial example (Adv1) having a perturbation length epsilon1 against x; conduct a search in a direction (Adv1-x) using the neural network; and to generate, using the neural network, a second adversarial example (Adv2) having a perturbation length epsilon2 based at least on an output of a search in the direction (Adv1-x).

System and method for max-margin adversarial training

Systems, methods, and computer readable media are described to train a compressed neural network with high robustness. The neural network is first adversarially pre-trained with both original data as well as data perturbed by adversarial attacks for some epochs, then 'unimportant' weights or filters are pruned through criteria based on their magnitudes or other method (e.g., Taylor approximation of the loss function), and the pruned neural network is retrained with both clean and perturbed data for more epochs.

Robust pruned neural networks via adversarial training

A method for acquiring measurements for a data structure corresponding to an array of variable includes: selecting a subset of elements from the data structure; measuring a sampled value for each of the selected subset of elements; storing each of the sampled values in a K-nearest neighbour (KNN) database and labelling the sampled value as certain; generating a predicted value data structure where each predicted element is generated as the value of its nearest neighbor based on the values stored in the KNN database; for each predicted element: retrieve the predicted element's X nearest neighbours for the sampled value in the KNN database, and when a value of the X nearest neighbours is the same as the predicted element, the predicted element is labelled as certain, otherwise the predicted element is labelled the values as uncertain; and repeating until all elements are labelled as certain.

Method and device for conducting measurements for an N-dimensional data structure

A platform for training deep neural networks using push-to-corner preprocessing and adversarial training. A training engine adds a preprocessing layer before the input data is fed into a deep neural network at the input layer, for pushing the input data further to the corner of its domain.

Growing community of inventors

Toronto, Canada

Ruitong Huang