Microsoft Technology Licensing, LLC

Khuram Shahid

Yifan Gong

Hosam Adel Khalil

Emilian Y Stoimenov

Guoli Ye

Kazuhito Koishida

Adina Magdalena Trufinescu

Cem Keskin

Bill Chau

Jaeyoun Kim

Wei Liu

Rui Zhao

Shuangyu Chang

Aleksandar Uzelac

Kaustubh Prakash Kalgaonkar

Piyush Behre

Edward C Lin

Adarsh Sridhar

Mahadevan Srinivasan

Anthony Phillip Stark

Che Zhao

Ivaylo Andreanov Enchev

Robert L Chambers

Thomas M Soemo

John Carroll Gordon

Vishwac Sena Kannan

David Keesu Kim

Nicholas Kibre

Shahin Shayandeh

Daniel J Hwang

Kshitiz Kumar

Daniel James Driscoll

Ali Reza Emami

Heiko Rahmel

Padma Varadharajan

Huaming Wang

Veljko Miljanic

Teng Yi

Solutions for custom display post processing (DPP) in speech recognition (SR) use a customized multi-stage DPP pipeline that transforms a stream of SR tokens from lexical form to display form. A first transformation stage of the DPP pipeline receives the stream of tokens, in turn, by an upstream filter, a base model stage, and a downstream filter, and transforms a first aspect of the stream of tokens (e.g., disfluency, inverse text normalization (ITN), capitalization, etc.) from lexical form into display form. The upstream filter and/or the downstream filter alter the stream of tokens to change the default behavior of the DPP pipeline into custom behavior. Additional transformation stages of the DPP pipeline perform further transforms, allowing for outputting final text in a display format that is customized for a specific user. This permits each user to efficiently leverage a common baseline DPP pipeline to produce a custom output.

Custom display post processing in speech recognition

Generally discussed herein are devices, systems, and methods for on-device detection of a wake word. A device can include a memory including model parameters that define a custom wake word detection model, the wake word detection model including a recurrent neural network transducer (RNNT) and a lookup table (LUT), the LUT indicating a hidden vector to be provided in response to a phoneme of a user-specified wake word, a microphone to capture audio, and processing circuitry to receive the audio from the microphone, determine, using the wake word detection model, whether the audio includes an utterance of the user-specified wake word, and wake up a personal assistant after determining the audio includes the utterance of the user-specified wake word.

On-device custom wake word detection

Generally discussed herein are devices, systems, and methods for custom wake word selection assistance. A method can include receiving, at a device, data indicating a custom wake word provided by a user, determining one or more characteristics of the custom wake word, determining that use of the custom wake word will cause more than a threshold rate of false detections based on the characteristics, rejecting the custom wake word as the wake word for accessing a personal assistant in response to determining that use of the custom wake word will cause more than a threshold rate of false detections, and setting the custom wake word as the wake word in response to determining that use of the custom wake word will not cause more than the threshold rate of false detections.

Wake word selection assistance architectures and methods

Systems and methods are disclosed for transitioning communication sessions across services. In one implementation, a first communication is received from a user via a first service. The first communication is associated with a communication session. The first communication is processed to identify a request with respect to a second service. The second service is initiated in response to the request.

Transitioning communication sessions across services

Generally discussed herein are devices, systems, and methods for wake word verification. A method can include receiving, at a server, a message from a device indicating that an utterance of a user-defined wake word was detected at the device, the message including (a) audio samples or features extracted from the audio samples and (b) data indicating the user-defined wake word, retrieving or generating, at the server, a custom decoding graph for the user-defined wake word, wherein the decoding graph and the static portion of the wake word verification model form a custom wake word verification model for the user-defined wake word, executing the wake word verification model to determine a likelihood that the wake word was uttered, and providing a message to the device indicating whether wake was uttered based on the determined likelihood.

Online verification of custom wake word

A server is provided, including a processor configured to execute a bot server program. The bot server program may receive from a computing device an input with an input type that includes one or more of speech and text. The bot server program may programmatically generate an output, wherein the output is generated based on the input. The bot server program may detect one or more output types capable of being output by the computing device and select an output type from a plurality of output types that may include speech and text. The selected output type may be an output type capable of being output by the computing device. The bot server program may modify the programmatically generated output to produce a modified output with the selected output type, and may convey the modified output to the computing device for output on a display and/or speaker.

Voice enabled bot platform

Computer systems, methods, and storage media for generating a continuous motion control using neurological data and for associating the continuous motion control with a continuous user interface control to enable analog control of the user interface control. The user interface control is modulated through a user's physical movements within a continuous range of motion associated with the continuous motion control. The continuous motion control enables fine-tuned and continuous control of the corresponding user interface control as opposed to control limited to a small number of discrete settings.

Continuous motion controls operable using neurological data

Computer systems, methods, and storage media for tailoring a user interface to a user according to a determined user state and a determined interface context corresponding to the determined user state. The user interface is tailored by modifying the format of at least a portion of the interface, including modifying the content, layout of the content, presentation sequence, or visual display of the interface. A user interface includes a selectable formatting object for controlling the formatting of the user interface and for generating feedback data for training an ensemble learning component to enable more effective predictive formatting changes.

Tailoring user interface presentations based on user state

Techniques are described for headlessly completing a task of an application in the background of a digital personal assistant. For example, a method can include receiving a voice input via a microphone. Natural language processing can be performed using the voice input to determine a user voice command. The user voice command can include a request to perform a task of the application. The application can be caused to execute the task as a background process without a user interface of the application appearing. A user interface of the digital personal assistant can provide a response to the user, based on a received state associated with the task, so that the response comes from within a context of the user interface of the digital personal assistant without surfacing the user interface of the application.

Headless task completion within digital personal assistants

Computer systems, methods, and storage media for changing the state of an application by detecting neurological user intent data associated with a particular operation of a particular application state, and changing the application state so as to enable execution of the particular operation as intended by the user. The application state is automatically changed to align with the intended operation, as determined by received neurological user intent data, so that the intended operation is performed. Some embodiments relate to a computer system creating or updating a state machine, through a training process, to change the state of an application according to detected neurological data.

Changing an application state using neurological data

Growing community of inventors

Seattle, WA, United States of America

Khuram Shahid