Apple Inc.

Liran Fishel

Erik K Norden

Assaf Metuki

Danny Gal

Christopher L Mills

Achmed Rumi Zahir

Lior Zimet

Inder M Sodhi

Sung Hee Park

Omri Flint

Ami Schwartzman

Nir Nissan

Fernando Alberto Mujica

Jaewon Shin

Per Hakan Hammarlund

Seungjin Lee

Roman Gindin

Wai Yu Trevor Tsang

Chuhan Min

David R Pope

Muge Wang

Etai Zaltsman

Zhimin Chen

Lukas Polok

Embodiments relate to generating keypoint descriptors of the keypoints. An apparatus includes a pyramid image generator circuit and a keypoint descriptor generator circuit. The pyramid image generator circuit generates an image pyramid from an input image. The keypoint descriptor generator circuit determines intensity values of sample points in the pyramid images for a keypoint and determines comparison results of comparisons between the intensity values of pairs of the sample points. The keypoint descriptor generator circuit generate bit values defining the comparison results for the keypoint, each bit value corresponding with one of the comparison results, and generate a sequence of the bit values defining an ordering of the comparison results based on importance levels of the comparisons, where the importance level of each comparison defines how much the comparison is representative of features. Bit values for comparisons having the lowest importance levels may be excluded from the sequence.

Configurable keypoint descriptor generation

Embodiments of the present disclosure relate to debugging of an accelerator circuit using a packet limit breakpoint. A vector circuit reads a subset of instruction packets from an instruction memory and receives a portion of input data from a data memory corresponding to the subset of instruction packets. The vector circuit executes a set of vector operations in accordance with multiple instruction packets from the subset using data from the received portion of input data identified in the multiple instruction packets to generate output data. A program counter control circuit coupled to the instruction memory triggers a breakpoint in a program stored in the instruction memory causing the accelerator circuit to stop executing remaining instruction packets in the program following the multiple instruction packets responsive to a number of instruction packets executed in the program from a time instant of an event reaching a predetermined number.

Debugging of accelerator circuit for mathematical operations using packet limit breakpoint

Embodiments of the present disclosure relate to sequentially loading keypoint descriptors of a previous image and comparing them with a plurality of keypoint descriptors of a current image. The keypoint descriptors of the previous image are stored and accessed from a system memory while the keypoint descriptors of the current image are stored and accessed from a local memory. Hence, the keypoint descriptors of the previous image are received only once at a descriptor match circuit while the keypoint descriptors of the current image are received multiple times for comparison against different keypoint descriptors of the previous image.

Single read of keypoint descriptors of image from system memory for efficient header matching

Embodiments relate to managing tasks that when executed by a neural processor circuit instantiates a neural network. A neural task manager circuit within the neural processor circuit can switch between tasks in different task queues. Each task queue is configured to store a reference to a task list of tasks for instantiating a neural network. Each task queue can also be assigned a priority parameter. While the neural processor circuit is executing tasks of a first task list and prior to completion of each task, the neural task manager circuit can switch between task queues according to the priority parameters for execution of tasks of a second task list by the neural processor circuit. The neural processor circuit includes one or more neural engine circuits that are configured to perform neural operations by executing the tasks assigned by the task manager.

Systems and methods for task switching in neural network processor

Techniques are disclosed that pertain to synchronizing power states between integrated circuit dies. A system includes an integrated circuit that includes a plurality of integrated circuit dies coupled together. A particular integrated circuit die may include a primary power manager circuit and one or more remaining integrated circuit dies include respective secondary power manager circuits. The primary power manager circuit is configured to issue a transition request to the secondary power manager circuits to transition their integrated circuit dies from a first power state to a second power state. A given secondary power manager circuit is configured to receive the transition request, transition its integrated circuit die to the second power state, and issue an acknowledgement to the primary power manager circuit that its integrated circuit die has been transitioned to the second power state. Techniques are further disclosed relating to managing latency tolerance events within a multi-die integrated circuit.

Blocking latency events in multi-die architecture

Embodiments relate to managing tasks that when executed by a neural processor circuit instantiates a neural network. The neural processor circuit includes neural engine circuits and a neural task manager circuit. The neural task manager circuit includes multiple task queues and a task arbiter circuit. Each task queue stores a reference to a task list of tasks for a machine learning operation. Each task queue may be associated with a priority parameter. Based on the priority of the task queues, the task arbiter circuit retrieves configuration data for a task from a memory external to the neural processor circuit, and provides the configuration data to components of the neural processor circuit including the neural engine circuits. The configuration data programs the neural processor circuit to execute the task. For example, the configuration data may include input data and kernel data processed by the neural engine circuits to execute the task.

Systems and methods for assigning tasks in a neural network processor

Embodiments of the present disclosure relate to selecting a subset of keypoint descriptors of two images for match operation based on their orientation angles indicated in headers of the keypoint descriptors. The keypoint descriptors in the two images are matched by first comparing their headers and then performing vector distance determination. During the header comparison operation, a header of a descriptor of a first image is compared only with headers of keypoint descriptors of a second image in a discrete orientation angle range corresponding to an orientation angle indicated by the header of the first image descriptor or keypoint descriptors of the second image in adjacent discrete orientation angle ranges. After the headers of the keypoint descriptors satisfying one or more matching criteria are determined, distance determination operations are performed between the keypoint descriptors while the remaining keypoint descriptors are discarded without determining their distances.

Filtering of keypoint descriptors based on orientation angle

Embodiments relate to a neural processor circuit with scalable architecture for instantiating one or more neural networks. The neural processor circuit includes a data buffer coupled to a memory external to the neural processor circuit, and a plurality of neural engine circuits. To execute tasks that instantiate the neural networks, each neural engine circuit generates output data using input data and kernel coefficients. A neural processor circuit may include multiple neural engine circuits that are selectively activated or deactivated according to configuration data of the tasks. Furthermore, an electronic device may include multiple neural processor circuits that are selectively activated or deactivated to execute the tasks.

Scalable neural network processing engine

Embodiments relate to extracting features from images, such as by identifying keypoints and generating keypoint descriptors of the keypoints. An apparatus includes a pyramid image generator circuit, a keypoint descriptor generator circuit, and a pyramid image buffer. The pyramid image generator circuit generates an image pyramid from an input image. The keypoint descriptor generator circuit processes the pyramid images for keypoint descriptor generation. The pyramid image buffer stores different portions of the pyramid images generated by the pyramid image generator circuit at different times and provides the stored portions of the pyramid images to the keypoint descriptor generator circuit for keypoint descriptor generation. When first portions of the pyramid images are no longer needed for the keypoint descriptor generation, the first portions are removed from the pyramid image buffer to provide space for second portions of the pyramid images that are needed for the keypoint descriptor generation.

Sliding window for image keypoint detection and descriptor generation

Synchronizing power state changes between multiple dies

Embodiments of the present disclosure relate to an accelerator circuit with a dynamic immediate values table (IVT). The accelerator circuit includes an instruction memory, a data memory, and a vector circuit with the IVT storing multiple immediate values at multiple entries. The vector circuit reads a subset of instructions from the instruction memory, each instruction including at least one corresponding pointer to at least one corresponding entry in the IVT. The vector circuit further receives a subset of input data from the data memory corresponding to the subset of instructions. The vector circuit performs a respective operation in accordance with each instruction from the subset of instructions using a corresponding data vector of the received subset of input data identified in each instruction and at least one corresponding immediate value from the IVT pointed by the at least one corresponding pointer to generate corresponding output data.

Accelerator circuit for mathematical operations with immediate values table

In an embodiment, a system includes rate limiter circuits corresponding to various agents that issue transactions in a virtual channel. At least one agent may be identified as a critical agent, and different rate limits (e.g., lower limits) may be selected for other agents when the critical agent is on than when the critical agent is off (e.g., higher limits).

Growing community of inventors

Raanana, Israel

Liran Fishel