Apple Inc.

Lsi Corporation

Cecile Marie Foret

Umesh Suresh Vaishampayan

Gaurav Kapoor

Kit-Man Wan

Francesco Rossi

Etienne Belanger

Xiaojin Shi

Marco Zuliani

Alexey Marinichev

Xiaozhong Yao

Sundararaman Hariharasubramanian

Benoit F Bazin

David R Pope

Derek R Kumar

Anthony Graham Sumpter

Jung Wan Kim

Harshavardhan P Gopalakrishnan

Albert Antony

William E Damon, Iii

Fabian P Wanner

The subject technology provides receiving a neural network (NN) model to be executed on a target platform, the NN model including multiple layers that include operations and some of the operations being executable on multiple processors of the target platform. The subject technology further sorts the operations from the multiple layers in a particular order based at least in part on grouping the operations that are executable by a particular processor of the multiple processors. The subject technology determines, based at least in part on a cost of transferring the operations between the multiple processors, an assignment of one of the multiple processors for each of the sorted operations of each of the layers in a manner that minimizes a total cost of executing the operations. Further, for each layer of the NN model, the subject technology includes an annotation to indicate the processor assigned for each of the operations.

Compiling models for dedicated hardware

The subject technology runs a compiled neural network (NN) model on a particular processor with multiple priority queues for executing different processes, the compiled NN model being assigned to a particular priority queue, and the compiled NN model includes context switch instructions that were previously inserted into a neural network (NN) model from which the compiled NN model was compiled. The subject technology determines that a particular context switch instruction has been executed by the particular processor. The subject technology determines that a different process is waiting to be executed, the different process being assigned to a different priority queue and the different process being a higher priority process than the running compiled NN model. In response to executing the particular context switch instruction, the subject technology performs a context switch to the different process assigned to the different priority queue when the different process is waiting to be executed.

The subject technology receives code corresponding to a neural network (NN) model and a set of weights for the NN model. The subject technology determines a set of layers that are mutable in the NN model. The subject technology determines information for mapping a second set of weights to the set of weights for the NN model. The subject technology generates metadata corresponding to the set of layers that are mutable, and the information for mapping the second set of weights to the set of weights for the NN model, wherein the generated metadata enables updating the set of layers that are mutable during execution of the NN model.

Mutable parameters for machine learning models during runtime

The subject technology receives code corresponding to a neural network (NN) model, the code including particular operations that are performed by the NN model. The subject technology determines, among the particular operations, a set of operations that are to be allocated to a cache of the electronic device that is to execute the NN model. The subject technology generates a set of cache indicators corresponding to the determined set of operations. The subject technology compiles the code and the generated set of cache indicators to provide a compiled binary for the NN model to execute on a target device.

Allocation of machine learning tasks into a shared cache

Embodiments relate a keypoint detection circuit for identifying keypoints in captured image frames. The keypoint detection circuit generates an image pyramid based upon a received image frame, and determine multiple sets of keypoints for each octave of the pyramid using different levels of blur. In some embodiments, the keypoint detection circuit includes multiple branches, each branch made up of one or more circuits for determining a different set of keypoints from the image, or for determining a subsampled image for a subsequent octave of the pyramid. By determining multiple sets of keypoints for each of a plurality of pyramid octaves, a larger, more varied set of keypoints can be obtained and used for object detection and matching between images.

Keypoint detection circuit for processing image pyramid in recursive manner

A method and apparatus of a device that manages a thermal profile of a device by selectively throttling graphics processing unit operations of the device is described. In an exemplary embodiment, the device monitors the thermal profile of the device, where the device executes a plurality of processes that utilizes a graphics processing unit of the device. In addition, the plurality of processes include a high priority process and a low priority process. If the thermal profile of the device exceeds a thermal threshold, the device decreases a first GPU utilization for the low priority process and maintains a second GPU utilization for the high priority process. The device further executes the low priority process using the first GPU utilization with the GPU and executes the high priority process using the second GPU utilization with the GPU.

Selective GPU throttling

An apparatus configured to process a digital video signal comprising an input circuit, a processing circuit and an encoder circuit. The input circuit may be configured to present a digital video signal comprising a plurality of frames. The processing circuit may be configured to detect scene changes in the digital video signal by analyzing (i) a current one of the plurality of frames and (ii) two or more other frames. The encoder circuit may be configured to generate an encoded signal in response to the digital video signal and the scene changes. The two or more other frames may comprise (i) a first window of frames that are processed before the current frame and (ii) a second window of frames that are processed after the current frame. The processing circuit may also detect the scene changes by analyzing changes between the first window and the second window.

Real time scene change detection in video sequences

Embodiments of the present invention relate to methods and systems that may detect cadences and duplicate fields in a video sequence, and correct the sequence to retrieve the original progressive content. Methods according to the present invention may compare consecutive fields of the same parity for similarity, and consecutive fields of the opposite parity for field and frame activity. By analyzing patterns in the similarity and activity of the fields, the cadence may be determined. The invention also provides methods that may be used to adjust for changes in the cadence, such as can result from scene changes in the video stream.

Pulldown correction for progressive display of audiovisual recordings

An apparatus configured to process a digital video signal comprising an input circuit, a processing circuit and an encoder circuit. The input circuit may be configured to present a digital video signal comprising a plurality of frames. The processing circuit may be configured to detect scene changes in the digital video signal by analyzing (i) a current one of the plurality of frames and (ii) two or more other frames. The encoder circuit may be configured to generate an encoded signal in response to the digital video signal and the scene changes.

Growing community of inventors

Palo Alto, CA, United States of America

Cecile Marie Foret