Microsoft Technology Licensing, LLC

Adi L Miller

Haim Somech

Ido Priness

Shira Weinberg

Roni Karassik

Michael Sterenberg

Shiraz Jitendra Cupala

Jaime Brooks Teevan

Hen Fitoussi

Amir Kantor

Lan Ye

Ido Cohn

Kevin Michael Carter

Shai Ifrach

Shlomi Maliah

Danny Avigdor

Yasemin Cenberoglu

Kumud Ranjan

Shyam Gupta

Gautam Jain

Xiao Yan Lu

Adi Diamant

Oded Nahir

Daniel Avigdor

Matthew Jonathan Gardner

Oded Vainas

Karen Master Ben-dor

Eleonora Shtotland

Gilad Gildin

Eyal Badash

Ofek David

Michal Bloch Ron

Adi Gerzi

Itamar Barkai

Faith Y Xu

Omri Gutman

A system for providing a personalized assistant within a network-based communication service includes one or more processors and a memory storage device storing instructions thereon. During a network-based communication session, the system receives a query from a computing device of a first communication session participant and processes the query by determining that a second communication session participant has shared content via a content sharing feature of the network-based communication service. In response, the system provides the query and at least a portion of the shared content as input to a model, which dynamically constructs a prompt for use as input with a generative language model. The system then presents the prompt as input to the generative language model, receives a response as output from the generative language model, and causes presentation of the response to be presented to the communication session participant.

Network-based communication session copilot

A system for providing a personalized assistant within a network-based communication session includes a processor and a memory storage device storing instructions. The system determines when a first communication session participant joins the network-based communication session after a threshold duration of time subsequent to the start time of the session. Upon determining the first participant has joined, the system obtains content associated with the session and creates request data for a pre-trained generative language model. The request data includes an instruction requesting a predetermined number of suggested utterances not present in the content, each utterance relating to one or more topics corresponding to the content. The system transforms the request data to a command based on a command template and provides the command to the generative language model. The system receives a response from the model, including the predetermined number of suggested utterances, and presents them to the communication session participant in a graphical user interface while the session is in session.

Systems and methods for surfacing natural language queries from one or more transcripts. An example method may include converting received audio to text, through automated speech recognition, to form a transcript of the audio, wherein the transcript includes text of the audio and identifications of speakers associated with portions of the text corresponding to utterances from the respective speakers; generating input signals based on at least the transcript; executing at least one of one or more heuristics or a trained machine-learning (ML) model, using the generated input signals as an input, to generate at least one of a suggested natural language query for searching the transcript or a key moment within the received audio; and causing at least one of the suggested natural language query or the key moment to be surfaced on one or more remote devices.

Suggested queries for transcript search

Technology is disclosed for programmatically determining, for a segment of a meeting recording, a user-specific adaptive playback speed, and generating a time-stretched segment playable at the adaptive playback speed. The adaptive playback speed is faster or slower than a default playback speed of the meeting recording. To determine the adaptive playback speed, this disclosure provides technologies to determine a playback data feature based on user-meeting data. The adaptive playback is generated based on the playback data feature. The segment is time-stretched to the adaptive playback speed to generate an updated meeting recording including the segment that is time-stretched and playable at the adaptive playback speed. In this manner, an updated meeting recording, specific to a user, and playable at an adaptive playback speed based on user-meeting data may reduce bandwidth associated with user's manually editing videos or rewinding playback, while improving user experience.

Personalized adaptive meeting playback

In some implementations, a first set of user activity data is received from a plurality of sensors where the first set of user activity data corresponds to a plurality of users. A task list of an objective is extracted from the received first set of user activity data based on patterns formed in the first set of user activity data in association with the plurality of users pursuing the objective. Based on determining a second set of user activity data indicates pursuit of the objective by a user, it is determined that at least a task of the task list is uncompleted by the user based on comparing the second set of user activity data to the task. Content corresponding to the task is provided to a user device associated with the user based on determining the task is uncompleted.

User objective assistance technologies

A response activity pattern may be ascertained from a set of computing devices obtained during a monitoring mode of operation. The monitoring mode of operation can be initiated when a user command is determined to be a new or unknown command. The response activity pattern may be used to generate a response profile indicating an activity to carry out using one or more user devices to perform a task associated with the user command. When an indication of a previously unknown user command to perform a task is received, the generated response profile can be used to perform the designed task by carrying out the activity using the one or more user device. In this way, the activity that should be carried out to perform the task can be learned based on the monitored user activity related to the one or more user devices.

Self-learning digital assistant

Intelligent agents (IA) for automatically generating responses to content within a communication session (CS) are disclosed. An IA is trained to target the responses to a user and the user's context within the CS. An IA receives CS content that includes natural language expressions encoding users' conversations and determines content features based on natural language models. The content features indicate intended semantics of the expressions. The IA identifies likely-relevant content to the targeted user, to generate a response for. Identifying such content includes determining a relevance of the content based on content features, a context of the CS, a user-interest model, and a content-relevance model. Identifying the likely-relevant content to respond to is based on the determined relevance of the content and relevance thresholds. Various responses to the identified portions of the content are automatically generated and provided based on a natural language response-generation model targeted to the user.

Virtual assistant for generating personalized responses within a communication session

Methods for providing enhanced services to users participating in communication sessions (CS), via a virtual assistant, are disclosed. One method receives content that is exchanged by users participating in the CS. The content includes natural language expressions that encode a conversation carried out by users. The method determines content features based on natural language models. The content features indicate intended semantics of the natural language expressions. The method determines a relevance of the content and identifies portions of the content that are likely relevant to the user. Determining the relevance is based on the content features, a context of the CS, a user-interest model, and a content-relevance model of the natural language models. Identifying the likely relevant content is based on the determined relevance of the content and a relevance threshold. A summary of the CS is automatically generated from summarized versions of the likely relevant portions of the content.

Virtual assistant providing enhanced communication session services

Aspects of the technology described herein maintain the privacy of confidential information to be communicated to a user through a computing device. The technology keeps confidential information private by assessing the privacy context of the communication. The privacy context can be determined by determining a privacy level of the information to be communicated and the privacy level of the environment into which the information is to be communicated. The privacy context can be used to select an appropriate communication channel for the information. The privacy context can also be used to determine whether all available content is shared or just a portion of it.

Privacy awareness for personal assistant communications

Methods, computer systems, computer-storage media, and graphical user interfaces are provided for facilitating content navigation, according to embodiments of the present invention. In one embodiment, a data structure is generated to organize data from content corresponding with an event at which the content is discussed. During the event, input is monitored, and an input for which to identify relevant content is detected. Based on the input, content that is relevant to the detected input is identified using the data structure by comparing at least a portion of the detected input with the data in the data structure. An indication of the relevant content is provided to trigger presentation of the relevant content, or a notification thereof.

Facilitating content navigation based on event context

Disclosed in some examples are methods, systems, machine-readable media, and devices to provide a personalized experience, by making user data available to a skill in a personal assistant service. In an example, a third-party skill such as an automated bot may obtain user data (such as user profile and contextual information at runtime) to personalize and adjust an interaction to the user. Operations to implement the techniques may include: providing a data interface of the personal assistant service to expose user data maintained by the personal assistant service, based on user data privacy settings and third-party skill permissions; receiving a request from the third-party skill, via the data interface, to obtain the user data to in connection with performing some action of the third-party skill; and providing the user data to the third-party skill in response to the request, based on the permission settings and the privacy settings.

Personalization of virtual assistant skills based on user profile information

In non-limiting examples of the present disclosure, systems, methods and devices for executing a command by a digital assistant in a group device environment are presented. A plurality of devices with digital assistants may be clustered for the duration of an event. One of the devices of the cluster may be assigned as an arbitrator device for the cluster. A user may issue a verbal command executable by a digital assistant of the cluster. The user that issued the verbal command may be identified via voice analysis. A determination may be made as to whether the verbal command corresponds to an intent to share content with a plurality of members of the cluster, or a specific member of the cluster, and a device of the cluster may be selected for executing a reply to the verbal command based on the determined intent and the executing device's presentation capabilities.

Identification and processing of commands by digital assistants in group device environments

Growing community of inventors

Ramat Hasharon, Israel

Adi L Miller