Microsoft Technology Licensing, LLC

Harish Kulkarni

David Abzarian

Todd L Carpenter

Akhilesh Kaza

Jonathan Campbell

Eric Norman Badger

Jacob Daniel Cohen

Meredith June Morris

Dmytro Rudchenko

Ann M Paradiso

Alexander John Fiannaca

Jay Beavers

Peter John Ansell

Jon Campbell

Mira E Shah

Fei Su

Austin Bradley Hodges

Salahuddin Khan

Narasimhan Raghunath

David Steeves

Michael K Fleming

Edward Bryan Cutrell

Jason William Fuller

Mark Myers

Sergey Solyanik

Seshagiri Panchapagesan

Cinthya R Urasaki

Xiaoyi Zhang

Kiley Rebecca Sobel

Dwayne Lamb

Trevor Armstrong

Systems and methods related to intelligent typing and responses using eye-gaze technology are disclosed herein. In some example aspects, a dwell-free typing system is provided to a user typing with eye-gaze. A prediction processor may intelligently determine the desired word or action of the user. In some aspects, the prediction processor may contain elements of a natural language processor. In other aspects, the systems and methods may allow quicker response times from applications due to application of intelligent response algorithms. For example, a user may fixate on a certain button within a web-browser, and the prediction processor may present a response to the user by selecting the button in the web-browser, thereby initiating an action. In other example aspects, each gaze location may be associated with a UI element. The gaze data and associated UI elements may be processed for intelligent predictions and suggestions.

Intelligent response using eye gaze

Systems and methods disclosed herein are related to an intelligent UI element selection system using eye-gaze technology. In some example aspects, a UI element selection zone may be determined. The selection zone may be defined as an area surrounding a boundary of the UI element. Gaze input may be received and the gaze input may be compared with the selection zone to determine an intent of the user. The gaze input may comprise one or more gaze locations. Each gaze location may be assigned a value according to its proximity to the UI element and/or its relation to the UI element's selection zone. Each UI element may be assigned a threshold. If the aggregated value of gaze input is equal to or greater than the threshold for the UI element, then the UI element may be selected.

Intelligent user interface element selection using eye-gaze

Systems and methods disclosed herein relate to assigning dynamic eye-gaze dwell-times. Dynamic dwell-times may be tailored to the individual user. For example, a dynamic dwell-time system may be configured to receive data from the user, such as the duration of time the user takes to execute certain actions within applications (e.g., read a word suggestion before actually selecting it). The dynamic dwell-time system may also prevent users from making unintended selections by providing different dwell times for different buttons. Specifically, on a user interface, longer dwell times may be established for the critical keys (e.g., 'close' program key, “send” key, word suggestions, and the like) and shorter dwell times may be established for the less critical keys (e.g., individual character keys on a virtual keyboard, spacebar, backspace, and the like).

Dynamic eye-gaze dwell times

Generation of expressive content is provided. An expressive synthesized speech system provides improved voice authoring user interfaces by which a user is enabled to efficiently author content for generating expressive output. An expressive synthesized speech system provides an expressive keyboard for enabling input of textual content and for selecting expressive operators, such as emoji objects or punctuation objects for applying predetermined prosody attributes or visual effects to the textual content. A voicesetting editor mode enables the user to author and adjust particular prosody attributes associated with the content for composing carefully-crafted synthetic speech. An active listening mode (ALM) is provided, which when selected, a set of ALM effect options are displayed, wherein each option is associated with a particular sound effect and/or visual effect. The user is enabled to rapidly respond with expressive vocal sound effects or visual effects while listening to others speak.

User interface for generating expressive content

Systems and methods disclosed herein relate to assigning dynamic eye-gaze dwell-times. Dynamic dwell-times may be tailored to the individual user. For example, a dynamic dwell-time system may be configured to receive data from the user, such as the duration of time the user takes to execute certain actions within applications (e.g., read a word suggestion before actually selecting it). The dynamic dwell-time system may also prevent users from making unintended selections by providing different dwell times for different buttons. Specifically, on a user interface, longer dwell times may be established for the critical keys (e.g., 'close' program key, 'send' key, word suggestions, and the like) and shorter dwell times may be established for the less critical keys (e.g., individual character keys on a virtual keyboard, spacebar, backspace, and the like).

A user's signature may be decomposed into one or more components. The components may be described using one or more control points. A user may sign with their eyes by focusing their gaze on a set of these control points that make up the signature. If the user's gaze is within a threshold distance from a control point the signature is validated. Moreover, by modifying the control points based upon the actual gaze location (which is within the threshold distance), the signature may be slightly modified. For example, the signature may be decomposed into one or more components as Bezier curves and the user may be asked to focus on each control point of each of the one or more Bezier curves. Modifying the control points of a Bezier curve slightly still produces a smooth curve, but introduces natural variations.

Signing with your eyes

The systems and methods described herein assist persons with the use of computers based on eye gaze, and allow such persons to control such computing systems using various eye trackers. The systems and methods described herein use eye trackers to control cursor (or some other indicator) positioning on an operating system using the gaze location reported by the eye tracker. The systems and methods described herein utilize an interaction model that allows control of a computer using eye gaze and dwell. The data from eye trackers provides a gaze location on the screen. The systems and methods described herein control a graphical user interface that is part of an operating system relative to cursor positioning and associated actions such as Left-Click, Right-Click, Double-Click, and the like. The interaction model presents appropriate user interfaces to navigate the user through applications on the computing system.

Controlling a computer using eyegaze and dwell

Techniques for text entry using gestures are disclosed. As disclosed, a camera may capture a frame and the face of the user can be detected therein. Landmarks can be aligned to the face in the captured frame. A left eye image and a right eye image may be extracted from the captured frame. The left eye image and the right image each may be resized and compared to a calibration template. A direction of eye gaze may be determined based upon the comparison. A character or word may be predicted based upon the determination of the direction of eye gaze and a known configuration of an eye gaze board (e.g., an E-tran board). The predicted character or word can be included as a part of a text-based message.

Automated E-tran application

Speech generating devices, communication systems, and methods for communicating using the devices and systems are disclosed herein. In certain examples, a communication system is configured to receive a generated communication, establish a connection between a speech generating device and a computing device subsequent to receipt of the generated communication, and transmit the generated communication to the computing device. In other examples, a computing device is configured to establish a connection with a speech generating device, and receive a transmission generated by the speech generating device following the connection, the transmission including previously generated communications or real-time communication segments or proxies. In other examples, a speech generating device is configured to establish a connection with one or more computing devices, receive one or more suggestions from at least one computing device during generation of the communication, and display a suggestion on the display device as a shortcut input key.

Facilitating awareness and conversation throughput in an augmentative and alternative communication system

Speech generating devices, communication systems, and methods for communicating using the devices and systems are disclosed herein. In certain examples, the speech generating device includes a display device and an input device configured to generate a communication to be displayed on the display device, wherein the speech generating device is configured to allow a user to select between playing the generated communication through a speaker and transmitting the generated communication via a communication network to a computing device separate from the speech generating device. In certain examples, the computing device may be designated with other computing devices of other conversation partners within a conversation group.

Directed personal communication for speech generating devices

Speech generating devices, communication systems, and methods for communicating using the devices and systems are disclosed herein. In certain examples, the speech generating device includes: a first display device positioned in a first direction, the first display device configured to face a user of the speech generating device; an input device configured to generate a communication to be displayed on the first display device; and a second display device positioned in a second direction different from the first direction, wherein the second display device is configured to display a social cue proxy or an emotional proxy of the generated communication.

Proxies for speech generating devices

Described is a technology by which a transient storage device or secure execution environment-based (e.g., including an embedded processor) device validates a host computer system. The device compares hashes of host system data against valid hashes maintained in protected storage of the device. The host data may be a file, data block, and/or memory contents. The device takes action when the host system data does not match the information in protected storage, such as to log information about the mismatch and/or provide an indication of validation failure, e.g., via an LED and/or display screen output. Further, the comparison may be part of a boot process validation, and the action may prevent the boot process from continuing, or replace an invalid file. Alternatively, the validation may take place at anytime.

Growing community of inventors

Redmond, WA, United States of America

Harish Kulkarni