Microsoft Technology Licensing, LLC

Oriana Riva

Douglas Christopher Burger

Suman Kumar Nath

Karin Strauss

Prashanth Hosabettu Mohan

Dimitrios Lymberopoulos

Earlence Tezroyd Fernandes

Jie Jerry Liu

Nissanka Arachchige Bodhi Priyantha

Christopher M White

Michael Gamon

Alice Jane Bernheim Brush

Jaron Zepel Lanier

Doug Burger

Stuart Schechter

Gerald Reuben DeJean

Tadayoshi Kohno

Franziska Roesner

Sujay Kumar Jauhar

Weiwei Yang

Timothy C Franklin

Eiji Hayashi

Naoki Otani

Mei Yang

Sri Raghu Malireddi

Jason Alan Kace

Yuanchun Li

Pratyay Banerjee

Kushal Arora

Jiajun Li

Yongjian Hu

Md Tanzirul Azim

Nitesh Mor

Gennady Pekhimenko

Temitope Oluwafemi

Shweti Mahajan

Sahisnu Mazumder

Jie Liu

Gerald Dejean

Systems and methods are described that are generally directed to generating a general task embedding representing task information. In examples, the generated task embedding may include predicted task information such that, rather being underspecified, the task embedding representative of the task may include additional specified information, where the task embedding can then be utilized in many different models and applications. Thus, task data may be received and at least a portion of the task data may be encoded using an encoder. Based on one or more outputs generated by the encoder and a type embedding associated with the task data, a task intent may be extracted or otherwise predicted based on the task data and one or more type encodings associated with the task data. The intent extractor may be trained on multiple auxiliary tasks with weak supervision that provide semantic augmentation to under-specified task texts.

Intent-based task representation learning using weak supervision

A computer implemented method includes accessing training data that includes images, associated alternative text, and proximately located text providing instructions describing a user interface. The images are paired with text captions derived from the proximately located text and image's alternative text, training a vision and language model in a self-supervised manner using language masking, region masking, and image-text alignment techniques on respective image region features and tokenized text captions. Fine-tuning of the vision and language model is performed to obtain a specialized model representing user interface elements and associated functions.

Self-supervised system for learning a user interface language

For a given input query specifying a task to be performed on a website, the correct sequence of actions (or UI script) is machine learned without having any previous knowledge about the website or the query. To learn the correct UI script, a task agent is created that performs multiple task agent runs comprising different sequences of actions of UI elements on the website (e.g., buttons, text fields, menus, and the like). The states of the webpages are monitored after each action of a UI element is performed. Tasklets are created that include the performed sequences of actions for each task agent as well as their assigned scores, and the correct UI script is chosen from the tasklets based on the scores (e.g., tasklet with the highest score).

Crawler of web automation scripts

Systems and methods for extracting and sharing application-related user data are disclosed. A method may include extracting in-app data for at least one of the plurality of apps running on a computing device, the in-app data including content consumed by a user while the at least one app is running, and/or at least one user action taken in connection with the content. Using an entity template associated with the app, a plurality of text strings within the in-app data are classified into at least one of a plurality of data types specified by the template. At least one user data item (UDI) may be generated by combining at least a portion of the classified plurality of text strings, the at least one UDI being accessible by a second app, an operating system running on the, a service of the operating system, and/or a service running on at least another device.

System and method for extracting and sharing application-related user data

The automatic generation of one or more task-oriented conversational bots is disclosed. Illustratively, systems and methods are provided that allow for tracing the interactions of one or more computing applications inclusive of the interaction with one or more programmatic elements of the one or more computing applications, interaction with the graphical user interface(s) of the one or more computing applications, and/or the operation of the one or more computing environments on which the one or more computing applications are executing to collect various state data. The state data can be illustratively graphed to show the overall execution paths of one or more functions/operations of the one or more computing applications for use in generating one or more instructions representative of a desired task-oriented conversational bot that can be operatively executed through one or more application program interfaces of the one or more computing applications.

Automatically generating conversational services from a computing application

Computer systems and methods for generating and interacting with a micro-service framework are provided. A micro-service corresponds to one or more deep link/API calls that carry out some particular function. A static analysis of an app is conducted, from one or more starting sources of the app to identify one or more valid and feasible execution paths, as well as corresponding input parameters within the app. Each valid execution path with corresponding input parameters represent a 'deep link' or 'API' for that app. The information regarding the deep link is collected and stored as a micro-service in a micro-service catalog. A micro-service framework is implemented that receives a micro-service request (i.e., a request that the micro-service be carried out on behalf of a computer user) from a UX client and executes that micro-service request via execution of the deep link.

Micro-service framework derived from third-party apps

Various technologies described herein pertain to prefetching content units. A prefetch request is transmitted to a server from a client device. The prefetch request includes data indicative of probabilities of slots for content units being available during an upcoming time period. The probabilities can be based on likely interaction with application(s) executed by the client device during the upcoming time period. Prefetched content units assigned to the client device for the upcoming time period can be received from the server responsive to the prefetch request. One or more of the prefetched content units can be served for display on a display screen of the client device during execution the application(s). Further, statuses of the prefetched content units can be monitored, and information that specifies a subset of the prefetched content units that are unlikely to be displayed on the display screen prior to corresponding deadlines for expiration can be transmitted.

Energy-efficient content serving

Various technologies described herein pertain to allocating content units to client devices. Content units can be requested from an exchange by a server. The content units can be received by the server from the exchange, where the content units each has a corresponding deadline for expiration. Further, an indication of a penalty can be transmitted to the exchange if one of the content units is not displayed by one or more client devices upon expiration of the corresponding deadline.

Techniques are described for validating stateful app links. Validation can be performed when stateful app links are created, activated, shared, or at other times. Validation can be performed to determine whether a stateful app link has a dependency on a resource external to the mobile application. Validation can also be performed to detect other issues, such as security issues, privacy issues, or other issues.

Validating stateful dynamic links in mobile applications

The privacy-preserving cookie generator implementations described herein create a privacy-preserving data structure (also called a privacy-preserving cookie herein) that is used to provide personalization for online services without user tracking. In some implementations the privacy-preserving cookie generator encodes a user profile (for example, based on a user's online activity) into a data structure that has naturally occurring noise and that efficiently supports noise addition. In one implementation a Bloom filter is used to create the encoded profile. Additional noise is injected into the encoded profile to create an obfuscated user profile in the form of a privacy-preserving data structure. The privacy-preserving data structure or cookie can be attached to an online service request and sent over a network to an online service provider which can use it fulfill the services request, providing a somewhat personalized result while the user's privacy is maintained.

Privacy-preserving cookies for personalization without user tracking

Examples of the disclosure enable a plurality of applications to be managed. Some examples include a mobile device that receives a template associated with an application, captures one or more user interactions with the application, and determines one or more user interests by comparing the one or more user interactions with a plurality of entities included in the template. Examples of the disclosure enable the mobile device to analyze the user interactions to provide valuable information about user interest, preference, and/or intent and provide tailored services to a user.

De-siloing applications for personalization and task completion services

This document relates to trend response management. One example can detect a trending topic and identify content associated with the trending topic. The example can take an action relating to the content that decreases a trend-induced processing spike and/or enhances a user search experience associated with the trending topic.

Trend response management

Systems and methods for creating and managing per-application profiles are disclosed. A method may include receiving input designating at least a first profile policy and a second profile policy. At least a first application profile and a second application profile may be created based on the received first profile policy and the second profile policy. An application of the plurality of applications may be associated with both the first application profile and the second application profile. A first storage partition and a second storage partition may be created within a storage space of the computing device. The storage space may be associated with the application. The first storage partition may store application data while the application is running under the first application profile. The second storage partition may store application data while the application is running under the second application profile.

System and method for using per-application profiles in a computing device

Growing community of inventors

Redmond, WA, United States of America

Oriana Riva