Microsoft Technology Licensing, LLC

Subramaniam Venkatraman Krishnan

Carlo Aldo Curino

Sriram S Rao

Konstantinos Karanasos

Christopher William Douglas

Yiwen Zhu

Rathijit Sen

Mostafa Elhemali

Kishore Chaliparambil

Kannababu Karanam

Chuan Liu

Fotios Psallidas

Yuan Yu

Alekh Jindal

Hiren Shantilal Patel

Shi Qiao

Avrilia Floratou

Ashvin Agrawal

Raghunath Ramakrishnan

Kwanghyun Park

Zhicheng Yin

Matteo Interlandi

Ya Lin

Rajeev Sudhakar Bhopi

Markus Weimer

Sreraman Narasimhan

Abhishek Roy

Joyce Yu Cahoon

Haitao Song

Wenjing Wang

Neetu Singh

Raymond Truong

XingYu Liu

Maria Alexandra Ciortea

Pratyush Rawat

Ivan Popivanov

Doris Suiyi Xin

Karla Jean Saur

Jonah Karpman

Nicholas Kent Glaze

Irwin Hollar Mcneely, Iii

Matthew Joseph Gleeson

Helen Mary Serr

MohammadHossein Namaki

Yinghui Wu

Andrew F Chung

Sudhir Darbha

Isha Tarte

Example solutions provide an artificial intelligence (AI) agent for pre-build configuration of cloud services in order to enable the initial build of a computational resource (e.g., in a cloud service) to minimize the likelihood of excessive throttling or slack. Examples leverage prior-existing utilization data and project metadata to identify similar use cases. The utilization data includes capacity information and resource consumption information (e.g., throttling and slack) for prior-existing computational resources, and the project metadata includes information for hierarchically categorization, to identify similar resources. A pre-build configuration is generated for the customer's resource, which the customer may tune based upon the customer's preferences for a cost and performance balance point.

AI agent for pre-build configuration of cloud services

Methods, systems, apparatuses, and computer-readable storage mediums described herein are directed to determining and recommending an optimal compute resource configuration for a cloud-based resource (e.g., a server, a virtual machine, etc.) for migrating a customer to the cloud. The embodiments described herein utilize a statistically robust approach that makes recommendations that are more flexible (elastic) and account for the full distribution of the amount of resource usage. Such an approach is utilized to develop a personalized rank of relevant recommendations to a customer. To determine which compute resource configuration to recommend to the customer, the customer's usage profile is matched to a set of customers that have already migrated to the cloud. The compute resource configuration that reaches the performance most similar to the performance of the configurations utilized by customers in the matched set is recommended to the user.

Cloud-based compute resource configuration recommendation and allocation for database migration

The description relates to executing an inference query relative to a database management system, such as a relational database management system. In one example a trained machine learning model can be stored within the database management system. An inference query can be received that applies the trained machine learning model on data local to the database management system. Analysis can be performed on the inference query and the trained machine learning model to generate a unified intermediate representation of the inference query and the trained model. Cross optimization can be performed on the unified intermediate representation. Based upon the cross-optimization, a first portion of the unified intermediate representation to be executed by a database engine of the database management system can be determined, and, a second portion of the unified intermediate representation to be executed by a machine learning runtime can be determined.

Query processing with machine learning

The cloud-based query workload optimization system disclosed herein the cloud-based query workloads optimization system receives query logs from various query engines to a cloud data service, extracts various query entities from the query logs, parses query entities to generate a set of common workload features, generates intermediate representations of the query workloads, wherein the intermediate representations are agnostic to the language of the plurality of the queries, identifies a plurality of workload patterns based on the intermediate representations of the query workloads, categorizes the workloads in one or more workload type categories based on the workload patterns and the workload features, and selects an optimization scheme based on the category of workload pattern.

Cloud based query workload optimization

An automated tuning service is used to automatically tune, or modify, the operational parameters of a large-scale cloud infrastructure. The tuning service performs automated and fully data/model-driven configuration based from learning various real-time performance of the cloud infrastructure. Such performance is identified through monitoring various telemetric data of the cloud infrastructure. The tuning service leverages a mix of domain knowledge and principled data-science to capture the essence of our cluster dynamic behavior in a collection of descriptive machine learning (ML) models. The ML models power automated optimization procedures for parameter tuning, and inform administrators in most tactical and strategical engineering/capacity decisions (such as hardware and datacenter design, software investments, etc.). Rich 'observational' models (models collected without modifying the system) are combined with judicious use of “fighting” (testing in production), allowing the tuning service to automatically configure operational parameters of a large cloud infrastructure for a broad range of applications.

Tuning large data infrastructures

A system enables tracking machine learning ('ML') model data provenance. In particular, a computing device is configured to accept ML model code that, when executed, instantiates and trains an ML model, to parse the ML model code into a workflow intermediate representation (WIR), to semantically annotate the WIR to provide an annotated WIR, and to identify, based on the annotated WIR and ML API corresponding to the ML model code, data from at least one data source that is relied upon by the ML model code when training the ML model. A WIR may be generated from an abstract syntax tree (AST) based on the ML model code, generating provenance relationships (PRs) based at least in part on relationships between nodes of the AST, wherein a PR comprises one or more input variables, an operation, a caller, and one or more output variables.

Tracking provenance in data science scripts

Described herein is a system and method for ranking and/or taking an action regarding execution of jobs of a shared computing cluster based upon predicted user impact. Information regarding previous executions of a plurality of jobs is obtained, for example, from job execution log(s). Data dependencies of the plurality of jobs are determined. Job impact of each of the plurality of jobs as a function of the determined data dependencies is calculated. User impact of each of the plurality of jobs as a function of the determined data dependencies, the calculated job impact, and time is calculated. The plurality of jobs are ranked in accordance with the calculated user impact. An action is taken in accordance with the ranking of the plurality of jobs. The action can include automatic scheduling of the jobs and/or providing information regarding the rankings to a user.

Provenance driven job relevance assessment

Embodiments are directed to progressively migrating source computer nodes where the source computer nodes perform a computer-implemented service. In one embodiment, a computer system determines that execution of the performed service is to be migrated from the source computer nodes to target computer nodes. The computer system groups the source computer nodes into multiple source subgroups, where each source subgroup includes at least one source computer node. The computer system then schedules creation of target subgroups of target nodes. These target subgroups include at least one source computer node and, themselves, correspond to a source subgroup. The computer system activates a first target subgroup corresponding to a first source subgroup, and deactivates the first source subgroup. In this manner, the first target subgroup replaces the first source subgroup. Still further, the target subgroups are scheduled to be created only after the first source subgroup has been deactivated.

Seamless cluster servicing

Growing community of inventors

Redmond, WA, United States of America

Subramaniam Venkatraman Krishnan