Microsoft Technology Licensing, LLC

Daniel J Hwang

Aleksandar Uzelac

Vishwac Sena Kannan

Sharath Viswanathan

Peter Gregory Davis

Juan (Lynn) Dai

Robert L Chambers

Jose Alberto Rodriguez

Joseph Bernard Tobens, Iii

Jeffrey C Fong

Shai Guday

Jeffery G Arnold

Lisa Joy Stifelman

Alice Jane Bernheim Brush

Thomas M Soemo

Sergio Paolantonio

Jonathan Campbell

Adina Magdalena Trufinescu

Khuram Shahid

Benjamin M Westbrook

Juan Dai

Yaser Khan

Ali Reza Emami

Lynn Dai

Jenny Kam

Dennis Mooney, Ii

Mouni Reddy

Priyanka Bhargavi Vegesna

Molly Rose Suver

Dhanya Krishnan Nair

Kenji Nakamura

John Sukkar

David Pinch

Zachary Thomas John Siddall

Zafeiria Anagnostopoulou

Dennis James Ii Mooney

Techniques are described herein that are capable of causing a control interface to be presented on a touch-enabled device based on a motion or absence thereof. A motion, such as a hover gesture, can be detected and the control interface presented in response to the detection. Alternatively, absence of a motion can be detected and the control interface presented in response to the detection. A hover gesture can occur without a user physically touching a touch screen of a touch-enabled device. Instead, the user's finger or fingers can be positioned at a spaced distance above the touch screen. The touch screen can detect that the user's fingers are proximate to the touch screen, such as through capacitive sensing. Additionally, finger movement can be detected while the fingers are hovering to expand the existing options for gesture input.

Presentation of a control interface on a touch-enabled device based on a motion or absence thereof

Described herein is a system and method leveraging infrastructure and client based location information in a hybrid positioning model. Location information can be received from a client device by a geospatial service component. For example, the location information can be calculated using fine time measurement (FTM) of IEEE 802.11mc and an access point. Positioning information can be determined based, at least in part, upon the received location information. The positioning information can include information regarding a physical environment of the client device. The positioning information can be provided to the client device.

Leveraging infrastructure and client based location information in a hybrid positioning model

An electronic device can receive user input via voice or text that includes tasks to be performed. A digital personal assistant infrastructure service can control to which registered action provider the task is assigned. Per-task action provider preferences can be stored. If a preferred action provider is not able to complete the task, the task can still be performed by a registered action provider that has appropriate capabilities. Machine learning can determine a user's preferences. Resource conservation and effective user interaction can result.

Extending digital personal assistant action providers

Techniques are described for headlessly completing a task of an application in the background of a digital personal assistant. For example, a method can include receiving a voice input via a microphone. Natural language processing can be performed using the voice input to determine a user voice command. The user voice command can include a request to perform a task of the application. The application can be caused to execute the task as a background process without a user interface of the application appearing. A user interface of the digital personal assistant can provide a response to the user, based on a received state associated with the task, so that the response comes from within a context of the user interface of the digital personal assistant without surfacing the user interface of the application.

Headless task completion within digital personal assistants

Techniques are described for discovering capabilities of voice-enabled resources. A voice-controlled digital personal assistant can respond to user requests to list available voice-enabled resources that are capable of performing a specific task using voice input. The voice-controlled digital personal assistant can also respond to user requests to list the tasks that a particular voice-enabled resource can perform using voice input. The voice-controlled digital personal assistant can also support a practice mode in which users practice voice commands for performing tasks supported by voice-enabled resources.

Discovering capabilities of third-party voice-enabled resources

An electronic device in a topology of interconnected electronic devices can listen for a wake phrase and voice commands. The device can control when and how it responds so that a single device responds to voice commands. Per-task device preferences can be stored for a user. If a preferred device is not available, the task can still be performed on a device that has appropriate capabilities. Machine learning can determine a user's preferences. Power conservation and effective user interaction can result.

Device arbitration for listening devices

A method for providing digital personal assistant responses may include receiving, by a digital personal assistant associated with a plurality of reactive agents, a user input initiating a dialog with the digital personal assistant within the computing device. In response to receiving the input, an operation mode of the computing device may be detected from a plurality of available operation modes. One of the plurality of reactive agents can be selected based on the received input. A plurality of response strings associated with the selected reactive agent can be accessed. At least one of the plurality of response strings is selected based at least on the operation mode and at least one hardware characteristic of the computing device. The selected at least one of the plurality of response strings is providing during the dialog, as a response to the user input.

Scaling digital personal assistant agents across devices

A method for updating language understanding classifier models includes receiving via one or more microphones of a computing device, a digital voice input from a user of the computing device. Natural language processing using the digital voice input is used to determine a user voice request. Upon determining the user voice request does not match at least one of a plurality of pre-defined voice commands in a schema definition of a digital personal assistant, a GUI of an end-user labeling tool is used to receive a user selection of at least one of the following: at least one intent of a plurality of available intents and/or at least one slot for the at least one intent. A labeled data set is generated by pairing the user voice request and the user selection, and is used to update a language understanding classifier.

Updating language understanding classifier models for a digital personal assistant based on crowd-sourcing

Techniques are described herein that are capable of increasing touch and/or hover accuracy on a touch-enabled device. For example, attribute(s) of a hand or a portion thereof (e.g., one or more fingers) may be used to determine a location on a touch screen to which a user intends to point. Such attribute(s) may be derived, measured, etc. For instance, a value corresponding to a distance between the hand/portion and the touch screen may be derived from a magnitude of a measurement of an interaction between the hand/portion and the touch screen. In another example, virtual elements displayed on the touch screen may be mapped to respective areas in a plane that is parallel (e.g., coincident) with the touch screen. In accordance with this example, receiving a touch and/or hover command with regard to an area in the plane may indicate selection of the corresponding virtual element.

Increasing touch and/or hover accuracy on a touch-enabled device

Disclosed herein are representative embodiments of tools and techniques for performing contextual searches using text determined based on digital-ink data. According to one exemplary technique, digital-ink data is received at a computing device and text is determined based on the digital-ink data. Additionally, by an application of the computing device, a contextual search is performed using the text.

Digital ink based contextual search

Display screen with graphical user interface

Growing community of inventors

Renton, WA, United States of America

Daniel J Hwang