Amazon Technologies, Inc.

Intel Corporation

Taemin Kim

Jindrich Zejda

Tobias Joseph Kastulus Edler Von Koch

Randy Renfu Huang

Drazen Borkovic

Ron Diamant

Richard John Heaton

Yida Wang

Animesh Jain

Yizhi Liu

Vinod Sharma

Paul S Diefenbaugh

Jeffrey T Huynh

Nithyananda Siva Jeganathan

Hongbin Zheng

Preston Pengra Briggs

Kyungtae Han

Sameer Abhinkar

A method of generating executable instructions for a computing system is provided. The method comprises: receiving a first set of instructions including a kernel of a first operator and a kernel of a second operator, the kernel of the first operator including instructions of the first operator and write instructions to a virtual data node, the kernel of the second operator including instructions of the second operator and read instructions to the virtual data node; determining, based on a mapping between the write instructions and read instructions, instructions of data transfer operations between the first operator and the second operator; and generating a second set of instructions representing a fused operator of the first operator and the second operator, the second set of instructions including the instructions of the first operator, the instructions of the second operator, and the instructions of the data transfer operations.

Performing hardware operator fusion

Methods of accelerating the execution of neural networks are disclosed. A description of a neural network may be received. A plurality of operators may be identified based on the description of the neural network. A plurality of symbolic models associated with the plurality of operators may be generated. For each symbolic model, a nested loop associated with an operator may be identified, a loop order may be defined, and a set of data dependencies may be defined. A set of inter-operator dependencies may be extracted based on the description of the neural network. The plurality of symbolic models and the set of inter-operator dependencies may be analyzed to identify a combinable pair of nested loops. The combinable pair of nested loops may be combined to form a combined nested loop.

Loop-oriented neural network compilation

Provided are systems and methods for synchronizing program code execution for a plurality of execution engines in an integrated circuit device. In some cases, the operation of one execution engine may be dependent on the operation of another execution engine. To accommodate this dependency, the instructions for the first execution engine can include a set-event instruction and the instructions for the second execution engine can include a wait-on-event instruction. The wait-on-event instruction can cause the second execution engine to wait for the first execution engine to reach the set-event instruction. In this way, the two execution engines can be synchronized around the data or resource dependency.

Synchronization of concurrent computation engines

Scheduling of the operations of an integrated circuit device such as a hardware accelerator, including scheduling of movement of data into and out of the accelerator, can be performed by a compiler that produces program code for the accelerator. The compiler can produce a graph that represents operations to be performed by the accelerator. Using the graph, the compiler can determine estimated execution times for the operations represented by each node in the graph. The compiler can schedule operations by determining an estimated execution time for set of dependent operations that depend from an operation. The compiler can then select an operation that has a shortest estimated execution time from among a set of operations and which has a set of dependent operations that has a longest estimated execution time as compared to other sets of dependent operations.

Compile-time scheduling

Provided are systems and methods for generating program code for an integrated circuit, where instructions in the code synchronize computation engines that support non-blocking instructions. In various examples, a computing device can receiving an input data set including operations to be performed by an integrated circuit device and dependencies between the operations. The input data set can include a non-blocking instruction, and an operation that requires that the non-blocking instruction be completed. The computing device can generate instructions for performing the operation including a particular instruction to wait for a value to be set in a register of the integrated circuit device. The computing device can further generate program code including the non-blocking instruction and the instructions for performing the operation, wherein the non-blocking instruction is configured to set the value in the register.

Synchronization of computation engines with non-blocking instructions

A hybrid display frame buffer for a display subsystem. An embodiment of an apparatus a first logic to split a video image into a first data portion and a second data portion; a display frame buffer including a first memory component having a first type of memory and a second memory component having a second type of memory, the first logic to write the first data portion to the first memory component and the second data portion to the second memory component; and a second logic to read the first data portion from the first memory component and the second data component from the second memory component, and to combine the first data portion and the second data portion to generate a combined video image.

Hybrid display frame buffer for display subsystem

Growing community of inventors

Portland, OR, United States of America

Taemin Kim