Rain Neuromorphics Inc.

University of Florida Research Foundation, Incorporated

Jack David Kendall

Juan Claudio Nino

Suhas Kumar

Alexander Almela Conklin

Nikita Gaur

Laura E Suarez

A bipartite memristive network and method of teaching such a network is described herein. In one example case, the memristive network can include a number of nanofibers, wherein each nanofiber comprises a metallic core and a memristive shell. The memristive network can also include a number of electrodes deposited upon the nanofibers. A first set of the number of electrodes can include input electrodes in the memristive network, and a second set of the number of electrodes can include output electrodes in the memristive network. The memristive network can be embodied as a bipartite memristive network and trained according to the method of teaching described herein.

Deep learning in bipartite memristive networks

Disclosed are systems and methods for performing efficient vector-matrix multiplication using a sparsely-connected conductance matrix and analog mixed signal (AMS) techniques. Metal electrodes are sparsely connected using coaxial nanowires. Each electrode can be used as an input/output node or neuron in a neural network layer. Neural network synapses are created by random connections provided by coaxial nanowires. A subset of the metal electrodes can be used to receive a vector of input voltages and the complementary subset of the metal electrodes can be used to read output currents. The output currents are the result of vector-matrix multiplication of the vector of input voltages with the sparsely-connected matrix of conductances.

Systems and methods for efficient matrix multiplication

A method for performing learning is described. A free inference is performed on a learning network for input signals. The input signals correspond to target output signals. The learning network includes inputs that receive the input signals, neurons, weights interconnecting the neurons, and outputs. The learning network is described by an energy for the free inference. The energy includes an interaction term corresponding to interactions consisting of neuron pair interactions. The free inference results in output signals. A first portion of the plurality of weights corresponding to data flow for the free inference. A biased inference is performed on the learning network by providing the input signals to the inputs and bias signals to the outputs. The bias signals are based on the target output signals and the output signals. The bias signals are fed back to the learning network through a second portion of the weights corresponding to a transpose of the first portion of the weights. At locations in the learning network, learning network equilibrium states are determined for the biased inference. The weights are updated based on the learning network equilibrium states.

Local training of neural networks

Learning algorithms for oscillatory memristive neuromorphic circuits are described. In one embodiment, a neuromorphic circuit learning network includes a number of neuromorphic circuit nodes, each including a recognition neuron unit and a generative neuron unit. The learning network further includes a plurality of neuromorphic circuit feedforward couplings between the recognition neuron units in the neuromorphic circuit nodes, and a plurality of neuromorphic circuit feedback couplings between the generative neuron units in the neuromorphic circuit nodes. The learning network also includes a learning controller configured to drive activity among the recognition neuron units and train the generative neuron units for learning in one mode and to drive activity among the generative neuron units and train the recognition neuron units for learning in another mode. Various deep learning algorithms can be implemented in the learning network. Two examples include the wake-sleep algorithm for unsupervised neural networks and target propagation.

Learning algorithms for oscillatory memristive neuromorphic circuits

A method for performing learning in a dissipative learning network is described. The method includes determining a trajectory for the dissipative learning network and determining a perturbed trajectory for the dissipative learning network based on a plurality of target outputs. Gradients for a portion of the dissipative learning network are determined based on the trajectory and the perturbed trajectory. The portion of the dissipative learning network is adjusted based on the gradients.

Learning in time varying, dissipative electrical networks

A memristive device and mechanisms for providing and using the memristive device are described. The memristive device includes a nanowire, a plurality of memristive plugs and a plurality of electrodes. The nanowire has a conductive core and an insulator coating at least a portion of the conductive core. The insulator has a plurality of apertures therein. The memristive plugs are for the apertures. At least a portion of each of the memristive plugs resides in each of the apertures. The memristive plugs are between the conductive core and the electrodes.

Memristive device

A method of training a learning network is described. The method includes generating a first estimate of a gradient for the learning network and generating subsequent estimates of the gradient using a feedback network. The feedback network generates improved perturbations for the subsequent gradient estimates. Gradient estimates include the first estimate of the gradient and the subsequent estimates of the gradient. The method also includes using the gradient estimates to determine weights in the learning network. The improved perturbations may include lower variance perturbations.

Machine learning using gradient estimate determined using improved perturbations

Disclosed are various embodiments of memristive devices comprising a number of nodes. Memristive fibers are used to form conductive and memristive paths in the devices. Each memristive fiber may couple one or more nodes to one or more other nodes. In one case, a memristive device includes a first node, a second node, and a memristive fiber. The memristive fiber includes a conductive core and a memristive shell surrounding at least a portion of the conductive core along at least a portion of the memristive fiber. The memristive fiber couples the first node to the second node through a portion of the memristive shell and at least a portion of the conductive core.

Memristive nanofiber neural networks

A system includes inputs, outputs, and nodes between the inputs and the outputs. The nodes include hidden nodes. Connections between the nodes are determined based on a gradient computable using symmetric solution submatrices.

Electrical networks using analytic loss gradients for design, analysis and machine learning

A method for performing learning is described. A free inference is performed on a learning network for input signals. The input signals correspond to target output signals. The learning network includes inputs that receive the input signals, neurons, weights interconnecting the neurons, and outputs. The learning network is described by an energy for the free inference. The energy includes an interaction term corresponding to interactions consisting of neuron pair interactions. The free inference results in output signals. A first portion of the plurality of weights corresponding to data flow for the free inference. A biased inference is performed on the learning network by providing the input signals to the inputs and bias signals to the outputs. The bias signals are based on the target output signals and the output signals. The bias signals are fedback to the learning network through a second portion of the weights corresponding to a transpose of the first portion of the weights. At locations in the learning network, learning network equilibrium states are determined for the biased inference. The weights are updated based on the learning network equilibrium states.

A memristive device is described. The memristive device includes a first layer having a first plurality of conductive lines, a second layer having a second plurality of conductive lines, and memristive interlayer connectors. The first and second layers differ. The first and second pluralities of conductive lines are each lithographically defined. The first and second pluralities of conductive lines are insulated from each other. The memristive interlayer connectors are memristively coupled with a first portion of the first plurality of conductive lines and memristively coupled with a second portion of the second plurality of conductive lines. The memristive interlayer connectors are thus sparsely coupled with the first and second pluralities of conductive lines. Each memristive interlayer connector includes a conductive portion and a memristive portion. The memristive portion is between the conductive portion and corresponding line(s) of the first plurality of conductive lines and/or the second plurality of conductive lines.

Lithographic memristive array

Memristive learning concepts for neuromorphic circuits are described. In one example case, a neuromorphic circuit includes a first oscillatory-based neuron that generates a first oscillatory signal, a diode that rectifies the first oscillatory signal, and a synapse coupled to the diode and including a long-term potentiation (LTP) memristor arranged in parallel with a long-term depression (LTD) memristor. The circuit further includes a difference amplifier coupled to the synapse that generates a difference signal based on a difference between output signals from the LTP and LTD memristors, and a second oscillatory-based neuron electrically coupled to the difference amplifier that generates a second oscillatory signal based on the difference signal. The circuit also includes a feedback circuit that provides a feedback signal to the LTP and LTD memristors based on a difference or error between a target signal and the second oscillatory signal.

Growing community of inventors

San Mateo, CA, United States of America

Jack David Kendall