The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Sep. 16, 2025
Filed:
Aug. 30, 2024
Dell Products L.p., Round Rock, TX (US);
Pedro Fernandez Orellana, Surfers Paradise, AU;
Qiang Chen, Shanghai, CN;
Dell Products L.P., Round Rock, TX (US);
Abstract
An apparatus comprises at least one processing device configured to determine, for a given batch to be executed utilizing at least one machine learning model, a number of requests to include based on an over-batching multiplier. The at least one processing device is also configured to allocate memory for processing the given batch, wherein an amount of memory allocated for at least requests in the given batch is less than that required for storage of a maximum output sequence length of the at least one machine learning model. The at least one processing device is further configured to execute the given batch utilizing the at least one machine learning model and, responsive to determining that at least one memory reallocation condition has been triggered during execution of the given batch, to adjust the allocation of the memory to one or more of the requests in the given batch.