The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
May. 26, 2026
Filed:
Nov. 19, 2024
Dell Products L.p., Round Rock, TX (US);
Shibi Panikkar, Bangalore, IN;
DELL PRODUCTS L.P., Round Rock, TX (US);
Abstract
A method facilitating memory-aware input batch processing for large language models includes monitoring, by a system including at least one processor, an amount of memory, utilized by a process in execution by a language model in association with processing a first batch of prompts, relative to an available amount of memory for the process in execution; in response to determining that the amount of memory utilized by the process in execution is at least a threshold proportion of the available amount of memory, removing, by the system, at least one prompt from the first batch of prompts according to a priority criterion, resulting in a second batch of prompts; and facilitating, by the system in response to the removing, restarting the process in execution with the second batch of prompts instead of the first batch of prompts.