Hewlett-packard Company

Nvidia Corporation

Edward H Gornish

Jack H Choquette

Sean Jeffrey Treichler

Ronny Krashinsky

Anne M Holler

Andrew Robert Kerr

Wei C Hsu

Kyrylo Perelygin

Alan Kaatz

Girish Bharambe

Apoorv Parle

Harold Carter Edwards

Konstantinos Kyriakopoulos

Maciej Piotr Tyrlik

Rishkul Kulkarni

Gokul Ramaswamy Hirisave Chandra Shekhara

Aditya Avinash Atluri

Balaji Krishna Yugandhar Atukuri

Bageshri Sathe

David Allan Berson

Haicheng Wu

Vijay Thakkar

Cody Addison

Aman Arora

James Player

Cris Cecka

Apparatuses, systems, and techniques to perform computational operations in response to one or more compute uniform device architecture (CUDA) programs. In at least one embodiment, one or more computational operations are to cause one or more other computational operations to wait until a portion of matrix multiply-accumulate (MMA) operations have been performed.

Application programming interface to wait on matrix multiply-accumulate

The present invention enables efficient pre-fetching of instructions. The present invention novelly determines a location for insertion of pre-fetch instructions earlier than in the past and in a cost effective manner. Therefore, the present invention introduces more control into the determination of when to initiate instruction pre-fetching than in the past. The present invention pre-fetches instructions accurately and launches instructions early enough to avoid cache miss latency. Also the present invention enables pre-fetching of instructions with the appropriate coverage. The present invention novelly generates pre-fetch instructions that have improved coverage over pre-fetching of the past by testing if the probability of a pre-fetch is cost effective and by determining whether the predicted size of a pre-fetched trace supports cost effective pre-fetching. The present invention assumes the existence of and utilizes the computer-based hardware capabilities of: a computer-based pre-fetch instruction that pre-fetches the cache line corresponding to a particular instruction address, and an augmentation to a computer-based branch instruction that can specify whether sequential instruction pre-fetching should be initiated at the target of a branch instruction.

Method, system, and apparatus to improve instruction pre-fetching on computer systems

There are two separate, yet related, prefetching strategies used for data references used having multiple strides, which typically occur in data references within nested loop structures. The first approach attempts to reverse one or more of the nested loops so that the strides of the reference are in the same direction. Once the loop or loops are reversed, data elements can be prefetched in the common loop direction. Preferably, the inner loops are reversed as compared with the outer loops, but this is not essential. The second approach is used where the data reference has multiple strides and the loops cannot be reversed. In this case, the prefetching method prefetches in the opposite direction of the innermost loop that surrounds the data reference. The second approach is used when the first approach cannot be used and where the strides of the reference have different directions and the inner loop is expected to iterate relatively few times. In such a case the second approach produces better results than the prior art.

Method of prefetching data for references with multiple stride directions

Growing community of inventors

Palo Alto, CA, United States of America

Edward H Gornish