Microsoft Technology Licensing, LLC

Harris Syed

Jonathan Reed Harris

Ashok Kuppusamy

Jaclyn Carley Knapp

Jesse Daniel Eskes Rusak

Richard Leigh Mains

Chad Steven Estes

Jayant Sivarama Krishnamurthy

Michael Francis Palermiti, Ii

Allison Anne O'Mahony

David Ernesto Heekin Burkett

Diego Hernan Carlomagno

Talon Edward Ireland

Jose Miguel Rady Allende

Andrew D Milligan

Andrew Paul McGovern

Sashi Raghupathy

Manoj Kumar Biswas

Viktoriya Taranov

Christopher Scott Dickens

Sermet Iskin

Sarjana Sheth

Ranjib Singh Badh

Max A Feingold

Jayaraman Kalyana Sundaram

Zainab Hakim

Saji Abraham

Jacob Andrew Sabulsky

Axel Andrejs

John Saxon Morrow

Tudor Buzasu Klein

Sergiy Gavrylenko

Donald F Box

Christian Liensberger

Henrik Frystyk Nielsen

Hen Fitoussi

John David Doty

Keith Coleman Herold

Jonathan E Hamaker

Erez Altus

Amy Harilal Rambhia

Pauline Chao

Ed Doran

Paul S Bloom

Andrew Paul Mcgovern

Various embodiments discussed herein enable client applications to be heavily integrated with a voice assistant in order to perform commands associated with voice utterances of users via voice assistant functionality and also seamlessly cause client applications to automatically perform native functions as part of executing the voice utterance. Such heavy integration also allows particular embodiments to support multi-modal input from a user for a single conversational interaction. In this way, client application user interface interactions, such as clicks, touch gestures, or text inputs are executed alternative or in addition to the voice utterances.

Voice assistant-enabled client application with user view context and multi-modal input support

Various embodiments discussed herein enable client applications to be heavily integrated with a voice assistant in order to both perform commands associated with voice utterances of users via voice assistant functionality and also seamlessly cause client applications to automatically perform native functions as part of executing the voice utterance. For example, some embodiments can automatically and intelligently cause a switch to a page the user needs and automatically and intelligently cause a population of particular fields of the page the user needs based on a user view context and the voice utterance.

Voice assistant-enabled client application with user view context

Techniques are described herein that are capable of providing extensibility with regard to a context-aware digital personal assistant. For instance, a description of a capability of a target (e.g., a software target) may be received. Examples of a target include but are not limited to an application (e.g., a software application), a service, a bot, and a website. In a first example, a digital personal assistant may be caused to perform operation(s) based at least in part on a context of a user. In a second example, whether the digital personal assistant performs operation(s) that are configured to be triggered by first aspect(s) of the context of the user is controlled based at least in part on second aspect(s) of the context of the user. The operation(s) include notifying the user of the capability and/or implementing the capability on behalf of the user.

Extensibility for context-aware digital personal assistant

A context aware digital assistant supported on devices such as smartphones, tablet computers, wearable computing devices, personal computers (PCs), game consoles, and the like is configured with a natural language interface that enables a user to launch searches for content using contextual references such as time, date, event, location, schedule, activity, contacts, or device. The user can thus use natural language to express the context that is applicable to the sought-after content rather than having to formulate a query that uses a specific syntax. The digital assistant can comprehensively search for the content across applications (i.e., both first and third party applications), devices, and services.

Contextual search using natural language

One or more techniques and/or systems are provided for dynamically maintaining user centric data. For example, a data provider app may have knowledge about user centric data associated with a user (e.g., a social network app may have contact information for a social network friend of the user). A user centric profile may be defined for the user centric data based upon information provided by the data provider app (e.g., a contact card may be generated for the social network friend). Responsive to receiving a request for the user centric profile from a requestor app (e.g., an event planning app), the user centric profile may be exposed to the user but not to the requestor app for security and/or privacy purposes. For example, an operating system may present at least some of the user centric profile within an operating system user interface.

User centric data maintenance

Example embodiments of the present invention provide a mechanism for allowing developers to define new interfaces and insert custom translational code that implements these new interfaces for legacy components that otherwise cannot communicate in a web service environment. These new interfaces are then exposed to the web service client in such a way that abstracts the web service client from the legacy components interfaces. The objects that implement the new interfaces are referred to herein as 'service surrogates' and the new interfaces will be commonly referred to as a, 'surrogate interfaces.' These surrogate interfaces may be initialized along with an instance of the service surrogate upon startup of an application that offers the services. In addition, a dynamic web service can simultaneously run along with the service surrogate for those services that may not need the features offered by the service surrogates.

Generating a dynamic web service and dynamic service surrogate for legacy application components

The present invention provides a mechanism that allows an administrative event to trigger or cause the generation of a dynamic web service during initialization of legacy application components. Similar to other types of static tooling approaches, the dynamic approach&#x2014;as supported herein&#x2014;uses available metadata describing an interface for communicating with legacy components. Rather than generating and manifesting the web service via tooling on a one time or occasional bases, however, example embodiments provide for the generation of the web service by the supporting infrastructure as a standard part of application initialization. Upon stopping the application, a termination sequence is provided that stops the corresponding dynamic web service and deletes the web service description used to generate the dynamic web service. Accordingly, every time the application is started the available metadata will need to be consulted and a consistent service will be built on-the-fly.

Dynamic service generation for legacy components

The present invention extends to methods, systems, and computer program products for creating proxies from service description metadata at runtime. A proxy creation request is received from a requesting application while the requesting application is executing at runtime. The proxy creation request requests creation of a proxy instance to facilitate compatible communication between the requesting application and another application. A proxy instance is created in response to the proxy creation request. Service description metadata describing the other application is accessed (e.g., through a request to a metadata service or retrieval from cache). The proxy instance is configured in accordance with the service description metadata describing the other application. Accordingly, the proxy instance can facilitate compatible communication between the requesting application and the other application to implement operations that the requesting application requests to be performed at the other application.

Creating proxies from service description metadata at runtime

Growing community of inventors

Redmond, WA, United States of America

Harris Syed