Language model decoding for search query completion

Abstract

A language model is used to generate autosuggestions to complete or revise a user's partial search query. An initial partial query is applied to the language model to generate query candidates for completing the search query. The language model may generate the query candidates as additional or alternate tokens for the partial search query. When the user revises the partial query, the previously-generated candidates can be re-used to reduce subsequent processing time for generating additional candidates. The previously-generated candidates are compared with the revised partial query to select which of the candidates to be re-used and expanded for generating additional tokens. Additional tokens can be generated in parallel for the previously-generated candidates or with model values from the previous generation, enabling the tokens to be generated effectively with reduced latency consistent with user expectations for search-related autosuggestions.