Applied Brain Research Inc.

Christopher David Eliasmith

Aaron Russell Voelker

Terrence Charles Stewart

Travis Crncich-DeWolf

Yichuan Tang

Daniel Halden Rasmussen

Peter Blouw

Trevor William Bekolay

Feng-Xuan Choo

Jean-Jacques Emile Slotine

Eric Gordon Hunsberger

Benjamin Jacob Morcos

Nachiket Ganesh Kapre

Brent Komer

Travis Crncich-Dewolf

Daniel Haiden Rasmussen

The present invention relates to methods and systems for using neural networks to simulate dynamical systems for purposes of solving optimization problems. More specifically, the present invention defines methods and systems that perform a process of 'synaptic descent' for performing “synaptic descent”, wherein the state of a given synapse in a neural network is a variable being optimized, the input to the synapse is a gradient defined with respect to this state, and the synapse implements the computations of an optimizer that performs gradient descent over time. Synapse models regulate the dynamics of a given neural network by governing how the output of one neuron is passed as input to another, and since the process of synaptic descent performs gradient descent with respect to state variables defining these dynamics, it can be harnessed to evolve the neural network towards a state or sequence of states that encodes the solution to an optimization problem.

Methods and systems for simulating dynamical systems via synaptic descent in artificial neural networks

The present invention relates to methods and systems for encoding and processing representations that include continuous structures using vector-symbolic representations. The system is comprised of a plurality of binding subsystems that implement a fractional binding operation, a plurality of unbinding subsystems that implement a fractional unbinding operation, and at least one input symbol representation that propagates activity through a binding subsystem and an unbinding subsystem to produce a high-dimensional vector representation of a continuous space.

Methods and systems for encoding and processing vector-symbolic representations of continuous spaces

The present invention relates to methods and systems for storing and querying database entries with neuromorphic computers. The system is comprised of a plurality of encoding subsystems that convert database entries and search keys into vector representations, a plurality of associative memory subsystems that match vector representations of search keys to vector representations of database entries using spike-based comparison operations, a plurality of binding subsystems that update retrieved vector representations during the execution of hierarchical queries, a plurality of unbinding subsystems that extract information from retrieved vector representations, a plurality of cleanup subsystems that remove noise from these retrieved representations, and one or more input search key representations that propagates spiking activity through the associative memory, binding, unbinding, cleanup, and readout subsystems to retrieve database entries matching the search key.

Methods and systems for storing and querying database entries with neuromorphic computers

The present invention relates to the digital circuits for evaluating neural engineering framework style neural networks. The digital circuits for evaluating neural engineering framework style neural networks comprised of at least one on-chip memory, a plurality of non-linear components, an external system, a first spatially parallel matrix multiplication, a second spatially parallel matrix multiplication, an error signal, plurality of set of factorized network weight, and an input signal. The plurality of sets of factorized network weights further comprise a first set factorized network weights and a second set of factorized network weights. The first spatially parallel matrix multiplication combines the input signal with the first set of factorized network weights called the encoder weight matrix to produce an encoded value. The non-linear components are hardware simulated neurons which accept said encoded value to produce a distributed neural activity. The second spatially parallel matrix multiplication combines said distributed neural activity with said second set of factorized network weights called the decoder weight matrix to produce an output signal.

Digital circuits for evaluating neural engineering framework style neural networks

Neural network architectures, with connection weights determined using Legendre Memory Unit equations, are trained while optionally keeping the determined weights fixed. Networks may use spiking or non-spiking activation functions, may be stacked or recurrently coupled with other neural network architectures, and may be implemented in software and hardware. Embodiments of the invention provide systems for pattern classification, data representation, and signal processing, that compute using orthogonal polynomial basis functions that span sliding windows of time.

Legendre memory units in recurrent neural networks

A method is described for designing systems that provide efficient implementations of feed-forward, recurrent, and deep networks that process dynamic signals using temporal filters and static or time-varying nonlinearities. A system design methodology is described that provides an engineered architecture. This architecture defines a core set of network components and operations for efficient computation of dynamic signals using temporal filters and static or time-varying nonlinearities. These methods apply to a wide variety of connected nonlinearities that include temporal filters in the connections. Here we apply the methods to synaptic models coupled with spiking and/or non-spiking neurons whose connection parameters are determined using a variety of methods of optimization.

Methods and systems for implementing dynamic neural networks

A method for implementing spiking neural network computations, the method including defining a dynamic node response function that exhibits spikes, where spikes are temporal nonlinearities for representing state over time; defining a static representation of said node response function; and using the static representation of the node response function to train a neural network. A system for implementing the method is also disclosed.

Methods and systems for implementing deep spiking neural networks

Methods, systems and apparatus that provide for perceptual, cognitive, and motor behaviors in an integrated system implemented using neural architectures. Components of the system communicate using artificial neurons that implement neural networks. The connections between these networks form representations&#x2014;referred to as semantic pointers&#x2014;which model the various firing patterns of biological neural network connections. Semantic pointers can be thought of as elements of a neural vector space, and can implement a form of abstraction level filtering or compression, in which high-dimensional structures can be abstracted one or more times thereby reducing the number of dimensions needed to represent a particular structure.

Methods and systems for artificial cognition

A system for generating and performing inference over graphs of sentences standing in directed discourse relations to one another, comprising a computer process, and a computer readable medium having computer executable instructions for providing: tree-structured encoder networks that convert an input sentence or a query into a vector representation; tree-structured decoder networks that convert a vector representation into a predicted sentence standing in a specified discourse relation to the input sentence; couplings of encoder and decoder networks that permit an input sentence and a 'query' sentence to constrain a decoder network to predict a novel sentence that satisfies a specific discourse relation and thereby implements an instance of graph traversal; couplings of encoder and decoder networks that implement traversal over graphs of multiple linguistic relations, including entailment, contradiction, explanation, elaboration, contrast, and parallelism, for the purposes of answering questions or performing dialogue transitions; and a spiking neural network implementation of the aforementioned system components.

Methods and systems for generating and traversing discourse graphs using artificial neural networks

Methods, systems and methods for designing a system that provides adaptive control and adaptive predictive filtering using nonlinear components. A system design is described that provides an engineered architecture. This architecture defines a core set of network dynamics that carry out specific functions related to control or prediction. The adaptation systems and methods can be applied to limited areas of the system to allow the system to learn to compensate for unmodeled system dynamics and kinematics. Two types of adaptive modules are described which are configured to account for the unmodeled system dynamics and kinematics.

Growing community of inventors

Waterloo, Canada

Christopher David Eliasmith