Microsoft Technology Licensing, LLC

Shengyu Fu

Neelakantan Sundaresan

Alexey Svyatkovskiy

Ying Zhao

Shao Kun Deng

Xiaoyu Liu

Shuo Zhang

Mark Alistair Wilson-Thomas

Christian Alma Bird

Colin Bruce Clement

Nan Duan

Miltiadis Allamanis

Shuai Lu

Vivian Julia Lim

Siyu Yang

Nina Wang

Roshanak Zilouchian Moghaddam

David Ellis Pugh

Kesavan Shanmugam

German David Obando Chacon

Shibani Basava

Srivatsn Narayanan

Dinesh Chandnani

Ram Kumar Donthula

Jonathan Daniel Keech

Zhu Chen

Andrew Christopher Neil

Jorge Banuelos

Geoffrey Staneff

Evgenia Steshenko

Matthew Sloan Theodore Evans

Zhongyan Guan

Jin Woo Jang

Jonathan Keith Simmons

Anqi Li

Jason Wang

David J Poeschl

Shwetha Srinath

Sai Tulasi Neppali

Siwei Li

George Joshua Michaels

Dongjiang You

Shubham Chandel

Vijay Upadya

David Poeschl

A hybrid inference system for a coding assistant utilizes a routing model to predict whether output generated by a large language model for a given prompt would be accepted by a user of the coding assistant. The routing model routes the prompt when the routing model indicates that the output generated by the large language model is likely to be accepted. The routing model routes the prompt to a local model when the output generated by the large language model is not likely to be accepted. The routing model is trained on the historical output generated by the large language model for various prompts and the acceptance or rejection of the output by users of the coding assistant.

Hybrid inference system for COGS reduction

A technique to prevent a prompt injection attack utilizes a security agent to sign a large language model prompt with a secret that is isolated from the user application or device that generates a user prompt. The secret is tailored for a specific user identifier and session identifier. The large language model is instructed to repeat the secret in each response. The security agent retrieves the response from the large language model and checks for the secret. When the secret is not part of the response, an error message is forwarded to the user application instead of the response.

Signing large language model prompts to prevent unintended response

A large language model, previously pre-trained on multiple source code modeling tasks, is pre-trained, through curriculum learning, to learn to predict a code execution trace given a source code program. The model is pre-trained using a variety of pre-training datasets consisting of pairs of a source code sample and a corresponding execution trace. The curriculum pre-training starts with a pre-training dataset of single line executions and adds in additional pre-training datasets with more increasing complex behaviors. The pre-training datasets include mutation-augmented source code samples and their corresponding execution traces.

Code execution trace generation with pre-trained large language model

A user query for information regarding data of a codebase is answered by a large language model given a prompt that includes examples of code segments from the codebase that are similar to the user query. The code segments from the codebase are associated with metadata that includes both natural language text and source code. The search for the examples of code segments from the codebase is based on embeddings of code segments and associated metadata that are closely similar to an embedding of the user query and context.

Code search for examples to augment model prompt

A deep learning model is pre-trained with a large-scale of unsupervised data of code review tasks in order to learn the relationships between code changes and a code review. The pre-trained deep learning model predicts a code review given a code diff hunk in a code diff format. The code diff hunk includes the changed code and its surrounding context. The pre-trained deep learning model may then be fine-tuned with supervised data in order to make predictions for several code review activities, such as, code change quality estimation and code refinement.

Pre-training for automating code review activities

A code completion tool uses a neural transformer model to generate candidate sequences to complete a line of source code. The neural transformer model is trained using a conditional language modeling objective on a large unsupervised dataset that includes source code programs written in several different programming languages. The neural transformer model is used within a beam search that predicts the most likely candidate sequences for a code snippet under development.

Multi-lingual line-of-code completion system

A retrieval-augmented neural transformer model with chunk cross-attention predicts a code review given a proposed source code change, represented as a code diff hunk, and a set of historical code review comments. The code diff hunk represents proposed edits to a source code snippet with its surrounding context that has not been changed. The historical code review comments are associated with code edits that are semantically similar to the proposed source code changes. The code diff hunk is partitioned into chunks which are used to find semantically similar historical code review comments. The set of historical code review comments is aggregated and used to guide the model in makings its predictions.

Code review comment generation via retrieval-augmented transformer with chunk cross-attention

A code adaptation mechanism automatically integrates the variable names of a pasted source code snippet into variable names defined in a pre-existing partial source code program. The variable names from the pasted source code snippet are replaced with anonymized values. A deep learning model predicts the most likely variable name from the pre-existing partial source code program to replace each anonymized value. The deep learning model is trained on numerous variable usage patterns from various source code programs to learn to predict the most likely mapping of an undefined variable name from the pasted source code snippet to a variable name in the pre-existing partial source code program thereby generating a syntactically and semantically correct program.

Code adaptation through deep learning

Edit automation enhancements may be implemented in source code editors and other text editors. Provisional selections that indicate user intentions are submitted to a suggestion generator with other edit context information, to improve the quality of generated text suggestions and reduce the cognitive load on users. A provisional selection may include a highlighted completion list entry, or document text targeted by a hovering cursor, or metainformation text targeted by the hovering cursor, for example. An inline grey text suggestion driven by provisional selection may be displayed simultaneously with completion list suggestions that were created without regard to provisional selection. Suggestions driven by provisional selection may be interleaved with existing document text. Suggestions may be accepted fully in one gesture, or in parts. Suggestions may be edited by a user before being accepted, driving further suggestion refinement. Multiple suggestions may be displayed simultaneously, reducing pressure on the suggestion generator.

Provisional selection drives edit suggestion generation

A natural language code search service provides idioms or frequently-occurring code patterns for a code fragment based on similar type usage and method/API invocation usage. The search service uses a data mining technique that mines code snippets found from various websites and code snippets generated from a neural model to detect idioms in the code snippets that were previously unknown and which can be reused. A search is initiated through a natural language query within a code development tool or application thereby avoiding the need to switch out of the current application to perform the search.

Natural language code search

A data mining technique is used to find large frequently-occurring source code patterns from methods/APIs that can be used in code development. Simplified trees that represent the syntactic structure and type and method usage of a source code fragment, such as a method, are mined to find closed and maximal frequent subtrees which represent the largest frequently-occurring source code patterns or idioms associated with a particular type and method usage. These idioms are then used in an idiom web service and/or a code completion system to assist users in the development of source code programs.

Frequent source code pattern mining

A code completion system predicts candidates to complete a code fragment with a tag name and/or an attribute name in source code written in a hierarchically-structured language. Candidates for predicting a tag name are based on a first-order tag Markov chain model generated from usage patterns of relationships of tag names found in a training dataset. Candidates for predicting an attribute name are based on a second-order attribute Markov chain model generated from usage patterns of sequences of attribute names associated with each tag name found in the training dataset.

Code completion for languages with hierarchical structures

A code completion tool uses machine learning models to more precisely predict the likelihood of the parameters of a method invocation. A score is computed for each candidate variable that is used to rank the viability of a variable as the intended parameter. The score is a weighted sum of a scope factor, an edit distance factor and a declaration proximity factor. The factors are based on a scope model, a method overload model, and a weight file trained offline on a training set of source code programs utilizing various method invocations.

Code completion of method parameters with machine learning

Growing community of inventors

Redmond, WA, United States of America

Shengyu Fu