Nuance Communications, Inc.

Herwig Häle

Andreas Neubacher

Attila Muszta

Christina Drexel

Miklós Pápai

Mert Öz

Peter Kozma

A multi-modal user interface is described that hides response latency delays. A graphical user interface (GUI) supports different user input modalities including low delay inputs which respond to user inputs without significant delay, and high latency inputs which have a significant response latency after receiving a user input before providing a corresponding completed response. The GUI accepts user inputs in a sequence of mixed input modalities independently of response latencies without waiting for responses to high latency inputs, and responds to each user input in the sequence as if each preceding user input in the sequence had been performed without response latencies.

Latency hiding techniques for multi-modal user interfaces

At design time of a graphical user interface (GUI), a software component (VUIcontroller) is added to the GUI. At run time of the GUI, the VUIcontroller analyzes the GUI from within a process that executes the GUI. From this analysis, the VUIcontroller automatically generates a voice command set, such as a speech-recognition grammar, that corresponds to controls of the GUI. The generated voice command set is made available to a speech recognition engine, thereby speech-enabling the GUI. Optionally, a GUI designer may add properties to ones of the GUI controls at GUI design time, without necessarily writing a voice command set. These properties, if specified, are then used at GUI run time to control or influence the analysis of the GUI and the automatic generation of the voice command set.

Adding speech capabilities to existing computer applications with complex graphical user interfaces

Growing community of inventors

Traiskirchen, Austria

Herwig Häle