International Business Machines Corporation

Other

Roy L Hoffman

Frank G Soltis

William G Kempke

Glen R Mitchell

Richard T Turner

John W McCullough

Larry Wayne Loen

Merle E Houdek

David Otto Lewis

Randy Lynn Egan

Richard Pervin King

Kurt Walter Pinnow

Daniel Manual Dias

Quentin G Schmierer

Thomas S Robinson

Christos Alkiviadis Polyzois

Robert W Collins

Nyles N Heise

David E Aakre

David N Moen

James J Pertzborn

Istvan S Kiss

A method is provided for parallel and cooperative processing of data in a system wherein a coordinator process cooperates with one or more agent processes to which portions of the data processing function is off loaded. The agent processes read and process the data and accumulate a partial result. Each agent process, responsive to statistics collected on the content of the data processed, returns a partial result of the processing to the coordinator process. These steps are repeated iteratively until the processing has been completed. In a specific application, the performance of data processing systems is improved by speeding up database group-by queries. The group-by operation processing is distributed between the host central processing unit (CPU) and the input/output (I/O) processors (IOPs). Essentially, the IOPs are sent group-by requests to be performed on a set of disk blocks (extents), along with a predicate for tuples to be selected for query. The IOPs build a hash table with entries of the group-by element and a running aggregation function (sum for example). The IOPs retrieve the extents, extract the records, select records using the predicate specified, enter the element in the hash table if it is not already there, and perform the corresponding aggregation function.

Distributed hash group-by cooperative processing

A real time data transformation and transmission apparatus transforms data from a first data device and transfers the transformed data to a second data device which need not have a data transfer rate consistent with the first data device. Data from the first data device is divided into blocks and is compressed by a compression device and written into a buffer. A controller controls the buffer to transmit compressed data to the second data device as a function of the data receiving rate of the second data medium provided that the buffer contains a predetermined amount of data. While the buffer is transmitting data, the compressor is compressing further blocks of data which are being written to the buffer such that the predetermined amount of data is stored in the buffer upon completion of the buffer transmitting a block of data. This ensures that complete blocks of data are transmitted to the second data medium at the data receiving rate of the second data medium.

Real time data transformation and transmission overlapping device

Storage serialization apparatus in a multiprocessor computer system enables multiple processors to concurrently execute instructions which access storage without materially affecting performance by keeping the amount of storage locked to a minimum, i.e., a page. The duration of serialization need be only for one instruction execution time and only instruction operands need be accommodated for serialization. Each storage request is intercepted by an associative register stack where there are two registers for each operand, one of the two being for operand page crossings. After a processor has locked access to an area of storage, execution of the instruction begins and all other processor are locked out but only with respect to that locked area. Other processors can access other storage areas during the instruction cycle. When the execution of the instruction completes, the processor releases the locked area of storage by invalidating the entries in its associative register array. A mechanism arbitrates and breaks possible deadlocks.

Multiprocessor storage serialization apparatus

A method and apparatus are provided to enable interruption of list processing operations in a computer system and to enable restart from the point of interruption. A mechanism, at a predetermined point of the list processing operation, operates to recognize occurrences of interrupting events. If any such events are present, a mechanism saves the status of the list processing operation, saves the identification of the task associated with instruction executing the list processing operation and locks the list or queue. After the interrupt is handled, a mechanism restores status, and unlocks the list or queue only when the identified task is active again and the instruction which had been executing the list processing operation is again executing.

Process and apparatus for interrupting and restarting sequential

Task dispatching for an asymmetric or symmetric multiprocessor system is provided where all the processors are dispatched from a single task dispatching queue. The workload, i.e. tasks, of the multiprocessor system is distributed to the available processors. Each processor includes a task dispatcher and a signal dispatcher. The signal dispatcher runs in a processor whenever a task dispatching element (TDE) is put on the task dispatching queue (TDQ) as a result of the task running in the processor. The signal dispatcher examines the TDEs enqueued on the TDQ and determines if any task dispatcher should be invoked, i.e. if any processor is running a lower priority task a task switch should occur. If so, it signals the selected processor to invoke its task dispatcher. After completing the task switch, the selected processor must invoke its signal dispatcher to determine if the task it had been performing should now be performed on some other processor in the multiprocessor system.

Multi-processor task dispatching apparatus

Control apparatus is responsive to CPU I/O commands for initiating chained I/O data transfers to cause virtual address translation (VAT) apparatus to translate a first virtual address to be used in the chained data transfer operation and load the translated (resolved) address in an I/O resolved address register reserved (unique) to the commanded I/O device connected to a shared I/O control unit and to repeat such an operation for each I/O device commanded by the CPU to do a data transfer and responsive to a command from the shared control unit indicating that one of the commanded I/O devices is ready for data transfer to become nonresponsive to further CPU I/O commands and cause the VAT to resolve a succession of virtual addresses for the data transfer and to load the resolved addresses into I/O resolved address registers shared for use by all of I/O devices whereby a data transfer operation can commence using the resolved address in the register unique to the I/O device which is first ready for data transfer and thereafter continue with data transfers using resolved addresses from the registers shared by all of the I/O devices but captured for use by the I/O device first ready for data transfer. Upon the data transfers being completed for that I/O device, the VAT resolves a succession of virtual addresses and loads them into the shared registers for use during data transfers by the next ready I/O device in response to a command from the shared I/O control unit.

Control apparatus for virtual address translation unit

Improved task handling apparatus for a computer system where the task dispatcher is selectively operable under instruction control for performing task queue selection and where the intertask communication mechanism can return a task dispatching element (TDE) to a non-prime task dispatching queue (TDQ) as well as to the prime TDQ. Whenever a TDE is returned to the prime TDQ, the task dispatcher makes a pre-emptive task switch. Also, if there are no task dispatching elements on the current non-prime TDQ, the task dispatcher switches to dispatch TDE's from the prime TDQ.

Task handling apparatus

An apparatus provides synchronization for page replacement control in a paged, virtual memory environment in which either the CPU or the I/O devices may pin and unpin pages to control their replacement by the paging supervisor. Pinning and unpinning of pages by the I/O devices occurs independently of pinning and unpinning performed by the CPU. Synchronization is achieved by means of a virtual address translation mechanism which is common to the CPU and the I/O devices. The virtual address translation mechanism includes a primary directory having entries for each page in main storage, with each entry containing a field in which the pinning and unpinning operations by the CPU and the I/O devices are registered. In particular, this field is a counter which is incremented when a page is pinned by either the CPU or an I/O device and decremented when a page is unpinned. Each page directory entry also includes a field for indicating that references to the page corresponding to a given page directory entry are invalid. When it is determined that a particular page is no longer in use and is eligible for replacement, the page directory entry is set to indicate that further references to the page are invalid. When an I/O device gives up a buffer page, that page is unpinned. Buffer pages newly acquired by the I/O device are pinned when they are acquired by the device and have become available in main storage.

Synchronizing mechanism for page replacement control

Virtual addressing apparatus for implementing a large virtual address in a computer system having narrow data paths, ALU, and local storage register arrays without requiring multiple passes. The virtual addressing apparatus stores the segment portion of a virtual address in a segment register and the offset portion of the virtual address in an offset register. To form a new virtual address, a new offset value is obtained by adding the displacement value given by the instruction in the instruction buffer register to the offset value stored in the offset register. The segment portion of the virtual address does not participate in the arithmetic operation for forming the new virtual address. The segment and offset portions are concatenated to form the new virtual address which is then translated to a main store address. Overflow detection circuitry in the ALU detects if an overflow out of the offset occurs as a result of the ALU operation for obtaining the new offset value. If an overflow is detected during the calculation of the new offset value, translation of the virtual address is subsequently aborted.

Virtual addressing apparatus employing separate data paths for segment

Address translation apparatus is provided for translating virtual addresses to real storage addresses and real storage addresses to virtual storage addresses. The address translation apparatus uses a page directory having a next real address and an associated virtual address ordered according to real addresses. This simplifies the manner in which the input/output (I/O) handles addressing in a virtual storage computer system. When the I/O device control mechanism needs to resolve the real I/O address register, it uses the contents of that register to index into the page directory to obtain a corresponding virtual address. The corresponding virtual address is incremented and converted to a real address which is used to index into the page directory. The virtual address taken from the page directory is then compared with the virtual address which had been incremented and translated. If the two compare then the real address which had been used to access the page directory is entered into a register so as to be available as a real main storage address. In actuality it is only a partial real main storage address and is concantenated with a byte identifier portion of the main storage address which requires no translation and which was a part of the original I/O real address for main storage.

Address translation apparatus

Task handling apparatus in a computer system is structured to be common to system control tasks, user tasks and I/O tasks. Although the task handling apparatus contains a task priority structure, all tasks are handled in the same manner, and there are no fixed interrupt levels for I/O tasks. There are N levels of priority, and N is variable. Each task is a server for a functional request. Task dispatching elements (TDE's) are enqueued in priority sequence on a task dispatching queue (TDQ). A task dispatcher functions to dispatch the highest priority TDE on the TDQ, if any, and to perform task switching. Intertask communication is accomplished by send message, send count, receive message and receive count mechanisms, and is coupled with task synchronization. Task synchronization is achieved by dequeueing and enqueueing TDE's on the TDQ. An active task becomes inactive dispatchable when a higher priority TDE is enqueued on the TDQ by send message or send count mechanisms. The active task becomes inactive waiting when a receive message or receive count is not satisfied. An inactive dispatchable task becomes the active task when it becomes the highest priority enqueued TDE on the TDQ by the receive message or receive count mechanisms. An inactive waiting task becomes either the active task or an inactive dispatchable task after being enqueued on the TDQ by the send message or send count mechanisms, depending upon whether it is the highest or other than the highest priority TDE on the TDQ.

Task handling apparatus for a computer system

A special directly executable instruction, Fetch Instruction Operand Address (FIOA) is accessed in response to encountering a complex non-directly executable instruction. Execution of the FIOA instruction causes generation of control signals for address calculation of the operands in the non-directly executable instruction by the same I phase hardware used by other directly executable instructions.

Growing community of inventors

Pine Island, MN, United States of America

Roy L Hoffman