Amazon Technologies, Inc.

Freescale Semiconductor,inc.

Qualcomm Incorporated

Nxp Usa, Inc.

Icontrol Networks, Inc.

Kun Xu

Ron Diamant

David B Kramer

Tommi M Jokinen

Ilya Minkin

Thomas A Volpe

Raymond Scott Whiteside

Paul J Dawes

David Leon Proft

Thuong Quang Truong

Paul Gilbert Meyer

Hien Le

Mark Banse

Jaya Prakash Subramaniam Ganasan

Patricio Kaplan

Mohammad El-Shabani

Jen-Tien Yen

Brian Robert Silver

Suresh Hariharan

Uday Shilton Udayaselvam

Micah Koch

Cesar Aaron Ramirez

Bohuslav Rychlik

Zheng Xu

Thomas Zeng

Jose Melanio Nunez

James Andrew Welker

Jason Edward Podaima

Thang Q Nguyen

John F Pillar

Marie Jeannette Sullivan

Gus P Ikonomopoulos

Christophe Avoinne

Michael Kardonik

Henry Wang

Manokanthan Somasundaram

Peter W Newton

Fnu Arun Kumar

Robert Serphillips

Tommi N Jokinen

Alwood Patrick Williams

Jaya Subramaniam Ganasan

Sharukh Shahajahan Shaikh

Hien Minh Le

An interconnect fabric in an integrated circuit (IC) device can be hierarchically partitioned into a set of peripheral sub-fabrics and a glue sub-fabric coupled to each of the peripheral sub-fabrics based on the physical design information associated with the IC device. Each of the peripheral sub-fabrics and the glue sub-fabric can be implemented using 2:1 hierarchical multiplexers based on an agglomerative clustering scheme. Physically aware design of the hierarchical multiplexer tree along with the optimal placement of the 2:1 multiplexers can provide smaller overall wire length and reduced routing congestion in the interconnect.

Physically aware design of interconnect fabric

Techniques for exchanging compressed gradient data within a distributed system are disclosed. A set of gradients are computed at a first worker node of the distributed system using a neural network model and a set of weights associated with the neural network model. Each of the set of gradients having a value less than a threshold is clipped, resulting in non-clipped data elements and clipped data elements. A mapping indicating which of the set of gradients correspond to non-clipped data elements and which of the set of gradients correspond to clipped data elements is generated. Compressed data is generated based on the non-clipped data elements. The mapping and the compressed data are transmitted from the first worker node to a second worker node of the distributed system.

Gradient compression for distributed training

To reduce the storage size of weight tensors and speed up loading of weight tensors from system memory, a compression technique can be employed to remove zero values from a weight tensor before storing the weight tensor in system memory. A sparsity threshold can be enforced to achieve a compression ratio target by forcing small weight values to zero during training. When the weight tensor is loaded from system memory, a direct memory access (DMA) engine with an in-line decompression unit can decompress the weight tensor on-the-fly. By performing the decompression in the DMA engine, expansion of the weight values back to the original weight tensor size can be carried out in parallel while other neural network computations are being performed by the processing unit.

Sparse machine learning acceleration

A technique for processing strong ordered transactions in a direct memory access engine may include retrieving a memory descriptor to perform a strong ordered transaction, and delaying the strong ordered transaction until pending write transactions associated with previous memory descriptors retrieved prior to the memory descriptor are complete. Subsequent transactions associated with memory descriptors following the memory descriptor are allowed to be issued while waiting for the pending write transactions to complete. Upon completion of the pending write transactions, the strong ordered transaction is performed.

Strong ordered transaction for DMA transfers

Techniques for distributing data associated with the weight values of a neural network model are described. The techniques can include performing computations associated with the neural network model in a neural network accelerator to generate data associated with weights of the neural network model. A multicast request packet is then generated to distribute the data. The multicast request packet may contain the data associated with the weights, and an address in a multicast address range of a peripheral bus multicast switch. The multicast request packet is sent to a port of the peripheral bus multicast switch, and in response, the peripheral bus multicast switch generates multiple packets containing the data from the multicast request packet and forwards them to multiple peripheral bus ports corresponding to other processing nodes of the system.

Distributive training with multicast

In one example, an apparatus comprises: a memory array having an array of memory elements arranged in rows and columns, each memory element being configured to store a data element; and a memory access circuit configured to: perform a row write operation to store a first group of data elements at a first row of the array of memory elements; perform a column read operation at a first column of the array of memory elements to obtain a second group of data elements; and perform a column write operation to store a third group of data elements at the first column of the array of memory elements to replace the second group of data elements.

Matrix transpose hardware acceleration

In one example, an apparatus comprises: a buffer memory; and a memory access circuit configured to: fetch, from a first memory, a set of first groups of data elements of a first matrix, each first group of data elements being stored at consecutive memory addresses at the first memory; based on a first configuration, store the set of first groups of data elements at consecutive memory addresses or at non-consecutive memory addresses at the buffer memory; based on a second configuration that defines a memory address offset, fetch a set of second groups of the data elements from the buffer memory, each second group of the data elements being stored at consecutive memory addresses of the buffer memory, each second group being separated by the memory address offset in the buffer memory; and store each fetched second group at consecutive addresses of a destination memory to form a second matrix.

A system and methods comprise a touchscreen at a premises. The touchscreen includes a processor coupled to a security system at the premises. User interfaces are presented via the touchscreen. The user interfaces include a security interface that provides control of functions of the security system and access to data collected by the security system, and a network interface that provides access to network devices. Network devices at the premises are coupled to the touchscreen. Application programming interfaces (APIs) are coupled to the processor and provide access to the plurality of network devices. A security server at a remote location is coupled to the touchscreen. The security server comprises a client interface through which remote client devices exchange data with the touchscreen and security system, a web application that accesses a first set of the APIs, and a server application that accesses a second set of the APIs.

Communication protocols in integrated systems

Techniques to reduce overhead in a direct memory access (DMA) engine can include processing descriptors from a descriptor queue to obtain a striding configuration to generate tensorized memory descriptors. The striding configuration can include, for each striding dimension, a stride and a repetition number indicating a number of times to repeat striding in the corresponding striding dimension. One or more sets of tensorized memory descriptors can be generated based on the striding configuration. Data transfers are then performed based on the generated tensorized memory descriptors.

Multidimensional and multiblock tensorized direct memory access descriptors

In one example, an apparatus comprises: a direct memory access (DMA) descriptor queue that stores DMA descriptors, each DMA descriptor including an indirect address; an address translation table that stores an address mapping between indirect addresses and physical addresses; and a DMA engine configured to: fetch a DMA descriptor from the DMA descriptor queue to the address translation table to translate a first indirect address of the DMA descriptor to a first physical address based on the address mapping, and perform a DMA operation based on executing the DMA descriptor to transfer data to or from the first physical address.

Direct memory access operation for neural network accelerator

To generate sequential addresses when multiple integrated circuit (IC) devices are accessing the same memory, an address token is sent along the IC devices communicatively coupled in a ring topology. The address token is first transferred along the ring topology during a memory reservation phase in which each IC device can set a corresponding memory request bit to indicate that the IC device has data to write to the memory. The modified address token is then transferred along the ring topology again during a memory access phase. During the memory access phase, each IC device that has data to write can perform a memory write operation using a sequential address determined from the contents of the address token.

Address generation for page collision prevention

To generate sequential addresses when multiple integrated circuit (IC) devices are accessing a memory region, an address token is sent along the IC devices communicatively coupled in a ring topology. The address token includes a data increment value for the memory region. When a device receives the address token, a memory write address is determined based on the data increment value and a base address corresponding to the memory region for the current write cycle. The IC device can perform a write operation using the memory write address if the device has data to write. The data increment value of the address token is then updated based on the number of data units being written in the current write cycle to the memory region by the IC device, and the updated address token is transmitted to the next IC device of the ring topology.

Address generation for page collision prevention in memory regions

To reduce direct memory access (DMA) overhead, a tensorized descriptor can be used to generate a series of memory descriptors to perform a series of DMA data transfers. The tensorized descriptor may include attributes such as a stride and a memory descriptor template, which can be used to generate the series of memory descriptors. Hence, instead of having to retrieve each of the memory descriptors to perform the series of DMA transfers, a single tensorized descriptor can be retrieved to perform a series of data transfers.

Tensorized direct memory access descriptors

A direct memory access (DMA) engine can be used to multicast data from system memory to a target memory for loading into an array. The DMA engine may include a controller that is configured to receive a data transfer request, and generate a set of write operations for the output interface. The set of write operations can include, for each of multiple partitions of the target memory, a write operation to write usable data from the multicast data to an address offset in the corresponding partition, and an additional write operation to write filler data from the multicast data to a null device address.

Data replication for accelerator

Growing community of inventors

Austin, TX, United States of America

Kun Xu