Microsoft Technology Licensing, LLC

Elnaz Nouri

Ryen William White

Robert Alexander Sim

Michael Gamon

Adam Fourney

Mark James Encarnacion

Nirupama Chandrasekaran

Bernd Ingo Plontsch

Sujay Kumar Jauhar

Russell Allen Herring, Jr

Omar Shaya

Benjamin Zorn

Andrew Donald Gordon

Vu Minh Le

Advait Sarkar

Carina Suzana Negreanu

Neil Toronto

Brian Paul Slininger

Sruti Srinivasa Ragavan

Shraddha Govind Barke

Carlos Toxtli Hernandez

Swaroop Ranjan Mishra

Christian Leopold Bejamin Poelitz

The indirect querying of models to determine capabilities possessed by the model. Such indirect queries take the form of model input that potentially includes a natural language input user data. Such model input is structured such that the output of the model is either not natural language at all, or else is natural language that is not semantically responsive to the natural language input. Nevertheless, the output is evaluated to estimate or determine the capability possessed by the model. Thus, models may be more fully utilized to their better potential.

Model capability extraction

Systems and methods relate to executing a task using a machine learning model based on prompt generation and collaborative interactions with a user. The machine language model generating a set of questions based on a task request. The user interactively answers the questions. A task processor generates a set of question-answer pairs based on the questions generated by the machine learning model and the answers given by the user. The machine learning model generates a task specific output based on the set of question-answer pairs. The machine learning model represents a large language model with deep learning. The simple question-and-answer prompts enable non-expert users to instruct the machine learning model with information that is sufficient to execute the task without overwhelming the users with the operations. The machine learning model leverages the answers to execute the task with accuracy, thereby providing efficacy of the prompting technique.

Prompt generation for guided custom machine learning collaboration

Aspects of the present disclosure relate to task template generation and social task discovery. In examples, a task template catalog comprises task templates, which may be automatically generated and/or user-submitted, among other examples. Task templates can be reviewed, shared, and curated within the task template catalog. A user may browse the task catalog or search the task catalog for task templates. Once the user selects a task template, a task is generated based on the task template and added to the user's task list. In some examples, aspects of a task template may be customized. For example, a task may comprise parametric or conditional subtasks, thereby enabling a user to further tailor the task template to his or her needs. Thus, the task catalog provides a starting point from which the user can author a task in a task management application.

Task templates and social task discovery

The present disclosure relates to systems and methods for an interactive, intelligent hub built around the completion of a task. This hub brings together resources, information, suggested steps, and other automated assistance to facilitate the completion of the task. AI-based assistance may indicate which steps can be completed by automated processes, and dispatch those processes, or suggest resources to assist in the completion of other steps. The hub displays the current status of the task, and lives until the completion of the task, or abandonment by the user.

Semi-autonomous intelligent task hub

Systems and methods are provided for automatically controlling a participant indication request based on a context of a meeting. The controlling of the participant indication request includes automatic lowering of a raised hand. A context determiner determines the context of the meeting based on meeting data including video, audio, background acoustic data, and chat messaging. The context determiner uses a global participant indication model for determining a context that is in commonly used among participants of the meeting. An individual participant indication model captures participant-specific rules of determining a context. A meeting state manager determines a meeting state based on the context. The meeting state includes a host presentation, a participant presentation, a conversation, and a polling. A participant indication controller automatically lowers the raised hand based on a combination of the determined context and the meeting state.

Automatically controlling participant indication request for a virtual meeting

In examples, a set of actions performed by a user is identified as an action sequence. If user performance of the same action sequence or similar action sequences exceeds a predetermined threshold, a recommendation to create a macro may be generated. The macro may have one or more associated triggers, such that it may be invoked using voice input or via a user interface, among other examples. A macro may have an associated context in which it applies. In some instances, a trigger used to invoke the macro comprises an indication as to such a context. For example, the macro may be invoked in the context of a document, such that one or more document parts are processed accordingly. As another example, the macro may be invoked to process multiple documents, as may be related in subject matter or associated with the same application.

Computer-generated macros and voice invocation techniques

In examples, a saliency metric may be determined for an instance of content and/or a location of a digital environment. Accordingly, a set of candidate content and/or locations may be ranked according to associated saliency metrics, such that content and/or an associated location with which to adapt the digital environment for a given user may be determined from a set of candidates. For example, the digital environment may be adapted to present a two-dimensional or a three-dimensional asset to a user. As another example, a game mechanic of the digital environment may be altered. In examples, content from another digital environment may be identified and used to adapt the digital environment, thereby incorporating external content. Thus, a saliency metric associated with a location or an instance of content may be used to programmatically generate a relative or absolute metric with which to evaluate and adapt aspects of the digital environment.

Saliency-based digital environment adaptation

Systems and methods are provided for determining importance and urgency of a task based on acoustic features of audio input associated with the task. The determining includes classifying the task into one or more classes associated with importance, urgency, and priority of the task. The classification may use a trained machine learning model of acoustic features and embedding for a neural network. The task classifier uses feature acoustics of either or both the foreground and background audio. The feature acoustics include a pitch, a tone, and a volume over a time duration of the audio input. A combination of the acoustic features determines a class associated with the task. The machine learning model includes a regression model of acoustic features over time and a model with embedding for a neural network.

Determination of task urgency based on acoustic features of audio data

The present disclosure provides systems and methods for user interaction and task completion using multiple devices. A set of devices may be used to perform a task, such that different devices may perform different steps of the task. A device management service may update state information at each device of the set, thereby enabling a user to interact with any of the computing devices to perform the task. A device management service may also automatically determine which device should be used by the user, based on task or step requirements, device characteristics, and device capabilities, among other examples. Thus, rather than being required to continue a task on the same device (even when the device is not well-suited for the current step or task), the user is provided with the option to use and, in some instances, is automatically transitioned to use, different devices within the set.

Growing community of inventors

Redmond, WA, United States of America

Elnaz Nouri