Language model-based conversational modeling

Abstract

A speech-processing system may include a language model configured to interact with a user in a dual mode in which the system processes an input while continuing to generate an output, or vice versa. Dual-mode operation may allow the user and system to engage in backchannel communication in which the user and/or system interjects speech while the other is speaking, but without an intent to interrupt or take a turn. For example, if the system is outputting speech and detects user speech, the system may continue generating/outputting speech while it processes the user speech to determine whether it likely represents backchanneling or an interruption. If the user speech likely represents backchanneling, the system may continue generating speech without yielding the turn. Similarly, the system may receive user speech and identify an opportunity for backchanneling before the user has finished speaking. The system may generate backchannel speech without taking a turn.