Microsoft Technology Licensing, LLC

Muskan Kukreja

Vijay Krishnan

Sandipan Ganguly

Shashidhar Rajashekara

Sandi Ganguly

Tianyi Chen

Ritu Singh

Michael C Cales

Usama Khurshid Haq

Hongwei Liang

Fnu Vijay Krishnan

Uma Shankar V Stanam

A method may include accessing, from a data store, telemetry data for an application, the telemetry data identifying actions executed by the application for a set of users of the application; retrieving an engagement index for the application using the telemetry data based on an engagement index profile, the engagement index profile identifying a selection of actions and a set of calculation dimensions; and presenting a user interface, the user interface including: a first portion identifying the selection of actions; a second portion identifying the set of calculation dimensions; a third portion identifying an original value of an action of the selection of actions with respect to a dimension in the set of calculation dimensions; and a fourth portion identifying a weighted composite value of the action with respect to set of calculation dimensions.

System for application engagement composite index

Non-limiting examples of systems, methods, and devices for automatic triaging of diagnostic failures for heterogeneous groups of tenants of a Software-as-a-Service, multi-tenant environment are disclosed herein. In an implementation, telemetry data for the heterogeneous groups of tenants is analyzed to classify individual tenant failures and detect the health status of the individual tenant. Tenant failures and/or tenant health statuses are filtered according to a threshold level. Anomalies having metrics that meet or exceed the threshold level are further analyzed to determine their priority (e.g., to a specific tenant). If the anomalies are known, then an existing entry for the anomaly is tagged and its priority may be changed. If the anomalies are unknown, then an entry is generated for the anomaly and prioritized. Tenants may be notified of a detected anomaly and may provide feedback. The feedback may be used to update triaging models.

Automatic triaging of diagnostics failures

A method of and system for performing pilot testing of a software program in an organization is carried out by collecting pilot testing data generated from a pilot testing of a software program run on one or more hardware assets in the organization, determining whether a sufficient amount of pilot testing data has been collected, and, when so, calculating one or more pilot test metrics from the collected data. The calculated pilot test metrics may then be compared to similar metrics in a target population to evaluate the software program.

Automatically performing and evaluating pilot testing of software

A system is disclosed, which includes a processor and a memory in communication with the processor. The memory includes executable instructions that, when executed by the processor, cause the processor to control the system to perform functions of collecting software asset information of software assets installed in hardware assets of an organization; determining popularity metrics of the software assets based on the collected software asset information; classifying, based on the popularity metrics, the software assets into a plurality of popularity groups; identifying, based on the classification, a pilot test software asset sample group; identifying, based on the pilot test software asset sample group, a pilot test hardware asset sample group; and deploying the new software asset to the pilot test hardware asset sample group for executing the pilot test of the new software asset.

Intelligent software asset classification for software update validation

In one example, a system includes an electronic processor configured to receive telemetry data originating from a plurality of client applications. The telemetry data includes data points associated with errors associated with one or more operations. The electronic processor also classifies the telemetry data based a plurality of classes of data, converts the plurality of classes of data into one or more metrics based on a plurality of dimensions, and aggregates the metrics for the classes of data by all the dimensions. The electronic processor accesses a predictive scoring model for a stored metric associated with a dimension of interest, determines a prediction error associated with the dimension of interest, detects an anomaly based on an item selected from the group consisting of the prediction error and a static threshold, and transmits an alert message, generates a bug report, and stores the bug report in a database.

Multi-variant anomaly detection from application telemetry

A device including a processor and memory is disclosed. The memory includes executable instructions that cause the processor to control the device to perform functions of identifying a pool of assets to be covered by a pilot test; receiving attributes associated with the identified assets in the pool; identifying a set of pilot test parameters associated with the pilot test; comparing the attributes of the assets with the identified set of pilot test parameters associated with the pilot test; based on a result of the comparison, providing a unique ranked order of the identified assets in the pool and selecting an asset from the unique ranked order to be placed in a pilot test group. When a stop condition is not satisfied, a footprint of the asset placed in the pilot test group is removed from the pool and the set of pilot test parameters.

Growing community of inventors

Redmond, WA, United States of America

Muskan Kukreja