The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Jul. 14, 2026
Filed:
Jul. 25, 2023
Google Llc, Mountain View, CA (US);
Sidharth Mudgal, Mountain View, CA (US);
Ahmad Beirami, New York, NY (US);
Jilin Chen, Cupertino, CA (US);
Alex Beutel, Brooklyn, NY (US);
Harish Ganapathy, Sunnyvale, CA (US);
Yaguang Li, Sunnyvale, CA (US);
Tao Wang, Sunnyvale, CA (US);
Yanping Huang, Mountain View, CA (US);
Trevor Strohman, Sunnyvale, CA (US);
GOOGLE LLC, Mountain View, CA (US);
Abstract
Implementations relate to reducing latency in generating and/or rendering a given stream of natural language (NL) based output generated using a large language model (LLM). Processor(s) of a system can: receive NL based input associated with a client device, generate the stream of NL based output utilizing the LLM that is responsive to the NL based input and that is for a given dialog context of an ongoing dialog, and cause the stream of NL based output to be rendered at the client device. Notably, the processor(s) can employ attribute classifier(s) and a multi-objective scorer to implement a blockwise controlled decoding technique in generating the stream of NL based output utilizing the LLM. By implementing the blockwise controlled decoding technique in generating the stream of NL based output utilizing the LLM, the processor(s) can reduce latency in generating and/or of the stream of NL based output generated utilizing the LLM.