Microsoft Technology Licensing, LLC

Apple Inc.

Dmytro Rudchenko

Timothy S Paek

Eric Norman Badger

Itai Almog

Parthasarathy Sundararajan

Akhilesh Kaza

Harish Kulkarni

Jason Anthony Grieves

Jacob Daniel Cohen

Tim Paek

Asela J Gunawardana

Gretchen Nash

Nate M Landes

Olivia Janik

Bongshin Lee

Amish Patel

Daniel Edward Ostrowski

Jonathan Campbell

Drew Elliott Linerud

Vishwas Kulkarni

Jerome Joseph Turner

Asela Jeevaka Ranaweera Gunawardana

Kozo Miyano

Masahiro Ami

Johnson T Apacible

Yutaka Suzue

Juan Dai

Minwoo Jeong

Keith P Avery

Peter John Ansell

Nikhil Devanur Rangarajan

Brandt M Westing

Timothy Seung Yoon Paek

Jamil Dhanani

Songming Ho

Drew Elliot Linerud

Harveen Kaur

Varun Maudgalya

Songming He

Gleb Krivosheev

Gleb G Krivosheev

Systems and methods related to intelligent typing and responses using eye-gaze technology are disclosed herein. In some example aspects, a dwell-free typing system is provided to a user typing with eye-gaze. A prediction processor may intelligently determine the desired word or action of the user. In some aspects, the prediction processor may contain elements of a natural language processor. In other aspects, the systems and methods may allow quicker response times from applications due to application of intelligent response algorithms. For example, a user may fixate on a certain button within a web-browser, and the prediction processor may present a response to the user by selecting the button in the web-browser, thereby initiating an action. In other example aspects, each gaze location may be associated with a UI element. The gaze data and associated UI elements may be processed for intelligent predictions and suggestions.

Intelligent response using eye gaze

Systems and methods disclosed herein relate to assigning dynamic eye-gaze dwell-times. Dynamic dwell-times may be tailored to the individual user. For example, a dynamic dwell-time system may be configured to receive data from the user, such as the duration of time the user takes to execute certain actions within applications (e.g., read a word suggestion before actually selecting it). The dynamic dwell-time system may also prevent users from making unintended selections by providing different dwell times for different buttons. Specifically, on a user interface, longer dwell times may be established for the critical keys (e.g., 'close' program key, “send” key, word suggestions, and the like) and shorter dwell times may be established for the less critical keys (e.g., individual character keys on a virtual keyboard, spacebar, backspace, and the like).

Dynamic eye-gaze dwell times

The subject technology receives, from a first sensor of a device, first sensor output of a first type. The subject technology receives, from a second sensor of the device, second sensor output of a second type, the first and second sensors being non-touch sensors. The subject technology provides the first sensor output and the second sensor output as inputs to a machine learning model, the machine learning model having been trained to output a predicted touch-based gesture based on sensor output of the first type and sensor output of the second type. The subject technology provides a predicted touch-based gesture based on output from the machine learning model. Further, the subject technology adjusts an audio output level of the device based on the predicted gesture, and where the device is an audio output device.

Machine-learning based gesture recognition

Systems and methods disclosed herein relate to assigning dynamic eye-gaze dwell-times. Dynamic dwell-times may be tailored to the individual user. For example, a dynamic dwell-time system may be configured to receive data from the user, such as the duration of time the user takes to execute certain actions within applications (e.g., read a word suggestion before actually selecting it). The dynamic dwell-time system may also prevent users from making unintended selections by providing different dwell times for different buttons. Specifically, on a user interface, longer dwell times may be established for the critical keys (e.g., 'close' program key, 'send' key, word suggestions, and the like) and shorter dwell times may be established for the less critical keys (e.g., individual character keys on a virtual keyboard, spacebar, backspace, and the like).

Embodiments are disclosed for a method of providing a user interface on a computing device. The method includes presenting a virtual keyboard on a display of the computing device, detecting input to the virtual keyboard. The method further includes, for each detected input, determining whether the input selects any of one or more delimiter keys, displaying a placeholder for the input responsive to the input not selecting any of the one or more delimiter keys, and receiving suggested candidate text from a word-level recognizer and replacing all currently displayed placeholders with the suggested candidate text responsive to the input selecting any of the one or more delimiter keys.

Display-efficient text entry and editing

An apparatus and method are disclosed for providing feedback and guidance to touch screen device users to improve text entry user experience and performance by generating input history data including character probabilities, word probabilities, and touch models. According to one embodiment, a method comprises receiving first input data, automatically learning user tendencies based on the first input data to generate input history data, receiving second input data, and generating auto-corrections or suggestion candidates for one or more words of the second input data based on the input history data. The user can then select one of the suggestion candidates to replace a selected word with the selected suggestion candidate.

User-centric soft keyboard predictive technologies

In a mobile device, the text entered by users is analyzed to determine a set of responses commonly entered by users into text applications such as SMS applications in response to received messages. This set of responses is used to provide suggested responses to a user for a currently received message in a soft input panel based on the text of the currently received message. The suggested responses are provided before any characters are provided by the user. After the user provides one or more characters, the suggested responses in the soft input panel are updated. The number of suggested responses displayed to the user in the soft input panel is limited to a total confidence value to reduce user distraction and to allow for easier selection. An undo feature for inadvertent selections of suggested responses is also provided.

String prediction

Described herein are various technologies pertaining to shapewriting. A touch-sensitive input panel comprises a plurality of keys, where each key in the plurality of keys is representative of a respective plurality of characters. A user can generate a trace over the touch-sensitive input panel, wherein the trace passes over keys desirably selected by the user. A sequence of characters, such as a word, is decoded based upon the trace, and is output to a display or a speaker.

Text entry using shapewriting on a touch-sensitive input panel

Embodiments present a game in which an ordered plurality of characters is presented for entry by a user with a touch screen, a physical keyboard, or other key input layout. The game advances to each successive character when the user presses the intended character or a character adjacent thereto. Contact areas are determined for each press, and in some embodiments the contact areas are overlaid on the keyboard. The contact areas are used to adjust user-specific touch models to improve text entry by the user. In some embodiments, the contact areas indicate areas for improvement by the user. Game completion statistics are calculated including speed and accuracy.

Text entry by training touch models

Display screen with animated graphical user interface

Growing community of inventors

Bellevue, WA, United States of America

Dmytro Rudchenko