Microsoft Technology Licensing, LLC

Andrei Nicolae

Rahul Nigam

Mark Raymond Gilbert

Parminder Pal Singh

Nidhi Verma

Vinod Mukundan Menon

Mark R Gilbert

In non-limiting examples of the present disclosure, systems, methods and devices for detecting and classifying service issues associated with a cloud-based service are presented. Operational event data for a plurality of operations associated with the cloud-based application service may be monitored. A statistical-based unsupervised machine learning model may be applied to the operational event data. A subset of the operational event data may be tagged as potentially being associated with a code regression, wherein the subset comprises a time series of operational event data. A neural network may be applied to the time series of operational event data, and the time series of operational event data may be flagged for follow-up if the neural network classifies the time series as relating to a positive code regression category.

Automated detection of code regressions from time-series data

Traditionally, when a feature is updated or a new feature is released, the feature undergoes internal testing and validation before external distribution. However, some features may receive proportionately less internal usage than customer usage reflected externally. Low internal usage of features can lead to weak telemetry data, which can allow code regressions (e.g., bugs) to go undetected until the features are released to customers. Accordingly, accelerated internal feature usage is provided to mirror external customer usage. Highly used features are dynamically identified and, any deficiencies in internal feature usage are identified. Tenant sites estimated to generate at least a portion of the deficiency in feature usage are identified. These sites may be migrated or replicated to internal validation rings to generate additional internal feature usage. By increasing internal testing and validation, the stability and reliability of feature releases are increased, thereby improving customer experience and satisfaction with the software product.

Accelerating pre-production feature usage

Disclosed herein is a system for automating the causality detection process when upgrades are deployed to different resources that provide a service. The resources can include physical and/or virtual resources (e.g., processing, storage, and/or networking resources) that are divided into different, geographically dispersed, resource units. To determine whether a root cause of a problem is associated with an upgrade event that has recently been deployed, a system is configured to use telemetry data to compute an upgrade-to-upgrade score that represents differences between two different upgrade events that are deployed to the same resource unit. The system is further configured to use telemetry data to compute an upgrade unit-to-unit score that represents differences between the same upgrade event being deployed to two different resource units. The scores can be used to output an alert, for an analyst, that signals whether a recently deployed upgrade event is the cause of a problem.

Growing community of inventors

Redmond, WA, United States of America

Andrei Nicolae