International Business Machines Corporation

Jose Guadalupe Delgado-Frias

Gerald George Pechanek

Stamatis Vassiliadis

Image processing for multimedia workstations is a computationally intensive task requiring special purpose hardware to meet the high speed requirements associated with the task. One type of specialized hardware that meets the computation high speed requirements is the mesh connected computer. Such a computer becomes a massively parallel machine when an array of computers interconnected by a network are replicated in a machine. The nearest neighbor mesh computer consists of an N&times;N square array of Processor Elements(PEs) where each PE is connected to the North, South, East and West PEs only. The diagonal folded mesh array processor, which is called Oracle, allows the matrix transformation operation to be accomplished in one cycle by simple interchange of the data elements in the dual symmetric processor elements. The use of Oracle for a parallel 2-D convolution mechanish for image processing and multimedia applications and for a finite difference method of solving differential equations is presented, concentrating on the computational aspects of the algorithm.

Massively parallel array processor

A massively parallel diagonal-fold mesh array processor provides a triangular diagonally folded mesh computer with the same functionality as a square mesh computer but with half the number of connection wires. The diagonal-fold mesh array processor is modified in this invention to provide a more general purpose processor node and to enhance the connectivity between the processing nodes while still providing the image processing and finite difference capabilities of the original structure. By repeatedly folding the triangular diagonal-fold array structure, processing elements are placed together which, with additional connections, allows the improvement in connectivity. This enhancement would be difficult to achieve in a standard mesh organization. The resultant folded structure maintains the functionality of the original mesh while expanding its capabilities. A bitonic sort example is presented which demonstrates the utility of the enhanced connectivity. The multiple folded array concept is applied to a six neighborhood hexagonal array demonstrating the general nature of the concept.

Massively parallel multiple-folded clustered processor mesh array

A massively parallel processor apparatus having an instruction set architecture for each of the N.sup.2 the PEs of the structure. The apparatus which we prefer will have a PE structure consisting of PEs that contain instruction and data storage units, receive instructions and data, and execute instructions. The N.sup.2 structure should contain 'N' communicating ALU trees, 'N' programmable root tree processor units, and an arrangement for communicating both instructions, data, and the root tree processor outputs back to the input processing elements by means of the communicating ALU trees. The apparatus can be structured as a bit-serial or word parallel system. The preferred structure contains N.sup.2 PEs, identified as PE.sub.column,row, in a N root tree processor system, placed in the form of a N by N processor array that has been folded along the diagonal and made up of diagonal cells and general cells. The Diagonal-Cells are comprised of a single processing element identified as PE.sub.i,i of the folded N by N processor array and the General-Cells are comprised of two PEs merged together, identified as PE.sub.i,j and PE.sub.j,i of the folded N by N processor array. Matrix processing algorithms are discussed followed by a presentation of the Diagonal-Fold Tree Array Processor architecture. The Massively Parallel Diagonal-Fold Tree Array Processor supports completely connected root tree processors through the use of the array of PEs that are interconnected by folded communication ALU trees.

Parallel diagonal-fold array processor

A massively parallel processor apparatus having an instruction set architecture for each of the N.sup.2 the PEs of the structure. The apparatus which we prefer will have a PE structure consisting of PEs that contain instruction and data storage units, receive instructions and data, and execute instructions. The N.sup.2 structure should contain 'N' communicating ALU trees, 'N' programmable root tree processor units, and an arrangement for communicating both instructions, data, and the root tree processor outputs back to the input processing elements by means of the communicating ALU trees. The apparatus can be structured as a bit-serial or word parallel system. The preferred structure contains N.sup.2 PEs, identified as PE.sub.column,row, in a N root tree processor system, placed in the form of a N by N processor array that has been folded along the diagonal and made up of diagonal cells and general cells. The Diagonal-Cells are comprised of a single processing element identified as PE.sub.i,j of the folded N by N processor array and the General-Cells are comprised of two PEs merged together, identified as PE.sub.i,j and PE.sub.j,i of the folded N by N processor array. Matrix processing algorithms are discussed followed by a presentation of the Diagonal-Fold Tree Array Processor architecture. The Massively Parallel Diagonal-Fold Tree Array Processor supports completely connected root tree processors through the use of the array of PEs that are interconnected by folded communication ALU trees.

Massively parallel diagonal-fold tree array processor

A parallel computer architecture supporting neural networks utilizing a novel method of separating a triangular array containing N processing elements on each edge into multiple smaller triangular arrays, each of dimension X and each representing a common building block processor group chip that can be interconnected for various size parallel processing implementations. The group chips are interconnected by a unique switching tree mechanism that maintains the complete connectivity capability and functionality possessed by the original triangular array of dimension N. For a given size K and X, K divisible by X, a triangular array containing K processor elements located on each edge of an equilateral triangular array is partitioned into K/X triangular arrays of dimension X and K(K-X)/2X.sup.2 square processor arrays of dimension X. An algorithm partitions a square array into two triangular arrays, each of dimension X. Assuming K=N and the chosen technology supports the placement of a triangular processor group chip of dimension X on a single chip, the final scalable parallel computing structure for N root tree processors utilizes N.sup.2 /X.sup.2 triangular processor group chips. The partitioning methodology creates a scalable organization of processor elements. An interconnection mechanism preserves the functionality of the original triangular array of dimension N in the implemented structure constructed of multiple triangular arrays of dimension X.

Scalable parallel group partitioned diagonal-fold switching tree

A Neural synapse processor apparatus having a neuron architecture for the synapse processing elements of the apparatus. The apparatus which we prefer will have a N neuron structure having synapse processing units that contain instruction and data storage units, receive instructions and data, and execute instructions. The N neuron structure should contain communicating adder trees, neuron activation function units, and an arrangement for communicating both instructions, data, and the outputs of neuron activation function units back to the input synapse processing units by means of the communicating adder trees. The apparatus can be structured as a bit-serial or word parallel system. The preferred structure contains N.sup.2 synapse processing units, each associated with a connection weight in the N neural network to be emulated, placed in the form of a N by N matrix that has been folded along the diagonal and made up of diagonal cells and general cells. Diagonal cells, each utilizing a single synapse processing unit, are associated with the diagonal connection weights of the folded N by N connection weight matrix and general cells, each of which has two synapse processing units merged together, and which are associated with the symmetric connection weights of the folded N by N connection weight matrix. The back-propagation learning algorithm is first discussed followed by a presentation of the learning machine synapse processor architecture. An example implementation of the back-propagation learning algorithm is then presented. This is followed by a Boltzmann like machine example and data parallel examples mapped onto the architecture.

Learning machine synapse processor system apparatus

Image processing for multimedia workstations is a computationally intensive task requiring special purpose hardware to meet the high speed requirements associated with the task. One type of specialized hardware that meets the computation high speed requirements is the mesh connected computer. Such a computer becomes a massively parallel machine when an array of computers interconnected by a network are replicated in a machine. The nearest neighbor mesh computer consists of an N.times.N square array of Processor Elements(PEs) where each PE is connected to the North, South, East and West PEs only. Assuming a single wire interface between PEs, there are a total of 2N.sup.2 wires in the mesh structure. Under the assumtion of SIMD operation with uni-directional message and data transfers between the processing elements in the meah, for example all PES transferring data North, it is possible to reconfigure the array by placing the symmetric processing elements together and sharing the north-south wires with the east-west wires, thereby reducing the wiring complexity in half, i.e. N.sup.2 without affecting performance. The resulting diagonal folded mesh array processor, which is called Oracle, allows the matrix transformation operation to be accomplished in one cycle by simple interchange of the data elements in the dual symmetric processor elements. The use of Oracle for a parallel 2-D convolution mechanish for image processing and multimedia applications and for a finite difference method of solving differential equations is presented, concentrating on the computational aspects of the algorithm.

Processing element for parallel array processor

A Neural synapse processor apparatus having a neuron architecture for the synapse processing elements of the apparatus. The apparatus which we prefer will have a N neuron structure having synapse processing units that contain instruction and data storage units, receive instructions and data, and execute instructions. The N neuron structure should contain communicating adder trees, neuron activation function units, and an arrangement for communicating both instructions, data, and the outputs of neuron activation function units back to the input synapse processing units by means of the communicating adder trees. The apparatus can be structured as a bit-serial or word parallel system. The preferred structure contains N.sup.2 synapse processing units, each associated with a connection weight in the N neural network to be emulated, placed in the form of a N by N matrix that has been folded along the diagonal and made up of diagonal cells and general cells. Diagonal cells, each utilizing a single synapse processing unit, are associated with the diagonal connection weights of the folded N by N connection weight matrix and general cells, each of which has two synapse processing units merged together, and which are associated with the symmetric connection weights of the folded N by N connection weight matrix. The back-propagation learning algorithm is first discussed followed by a presentation of the learning machine synapse processor architecture. An example implementation of the back-propagation learning algorithm is then presented. This is followed by a Boltzmann like machine example and data parallel examples mapped onto the architecture

A neural network architecture consisting of input weight multiplications, product summation, neural state calculations, and complete connectivity among the neuron processing elements. Neural networks are modelled using a sequential pipelined neurocomputer producing high performance with minimum hardware by sequentially processing each neuron in the completely connected network model. An N neuron network is implemented using multipliers, a pipelined adder tree structure, and activation functions. The activation functions are provided by using one activation function module and sequentially passing the N input product summations sequentially through it. One bus provides N.times.N communications by sequentially providing N neuron values to the multiplier registers. The neuron values are ensured of reaching corresponding multipliers through a tag compare function. The neuron information includes a source tag and a valid signal. Higher performance is provided by connecting a number of the neurocomputers in a parallel.

SPIN: a sequential pipeline neurocomputer

A scalable flow virtual learning neurocomputer system and appratus with a scalable hybrid control flow/data flow employing a group partitioning algorithm, and a scalable virtual learning architecture, synapse processor architecture mapping, inner square folding and array separation, with capability of back propagation for virtual learning. The group partitioning algorithm creates a common building block of synapse processors containing their own external memory. The processor groups are used to create a general purpose virtual learning machine which maintains complete connectivity with high performance. The synapse processor group allows a system to be scalable in virtual size and direct execution capabilities. Internal to the processor group, the synapse processors are designed as a hybrid control flow/data flow architecture with external memory access and reduced synchronization problems.

Scalable flow virtual learning neurocomputer

The Pyramid Learning Architecture Neurocomputer (PLAN) is a scalable stacked pyramid arrangement of processor arrays. There are six processing levels in PLAN consisting of the pyramid base, Level 6, containing N.sup.2 SYnapse Processors (SYPs), Level 5 containing multiple folded Communicating Adder Tree structures (SCATs), Level 4 made up of N completely connected Neuron Execution Processors (NEPs), Level 3 made up of multiple Programmable Communicating Alu Tree (PCATs) structures, similar to Level 5 SCATs but with programmable function capabilities in each tree node, Level 2 containing the Neuron Instruction Processor (NIP), and Level 1 comprising the Host and user interface. The simplest processors are in the base level with each layer of processors increasing in computational power up to a general purpose host computer acting as the user interface. PLAN is scalable in direct neural network emulation and in virtual processing capability. Consequently, depending upon performance and cost trade-offs, the number of physical neurons N to be implemented is chosen. A neural network model is mapped onto Level 3 PCATs, Level 4 NEPs, Level 5 SCATs, and Level 6 SYPs. The Neuron Instruction Processor, Level 2, controls the neural network model through the Level 3 programmable interconnection interface. In addition, the NIP level controls the high speed and high capacity PLAN I/O interface necessary for large N massively parallel systems. This discussion describes the PLAN processors attached to a Host computer and the overall control of the pyramid which constitutes the neurocomputer system.

Pyramid learning architecture neurocomputer

The architectures for a scalable neural processor (SNAP) and a Triangular Scalable Neural Array Processor (T-SNAP) are expanded to handle network simulations where the number of neurons to be modeled exceeds the number of physical neurons implemented. This virtual neural processing is described for three general virtual architectural approaches for handling the virtual neurons, one for SNAP and one for TSNAP, and a third approach applied to both SNAP and TSNAP.

Growing community of inventors

Endwell, NY, United States of America

Jose Guadalupe Delgado-Frias